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Section  1.0  INTRODUCTION 


Contracting  is  the  formalized  method  of 
procuring  an  identified  product  under 
specified  conditions.  The  process  of  con¬ 
summating  and  managing  ground  system 
software  procurement  involves  specifying 
complex  software  and  hardware/software 
interfaces,  evaluating  competing  compa¬ 
nies  and  their  proposed  systems  and  man¬ 
aging  technical  requirements,  schedules, 
and  costs.  This  guidebook  discusses 
these  processes  from  early  planning 
through  contract  fulfillment,  as  these 
processes  specifically  apply  to  software 
for  Training  Simulators  (TS)  and  Auto¬ 
matic  Test  Equipment  (ATE). 

1.1  PURPOSE 

It  is  the  purpose  of  this  guidebook  to 
assist  USAF  procurement  engineering  per¬ 
sonnel  in  the  acquisition  of  Training 
Simulators  and  Automatic  Test  Equipment 
and  to  ensure  uniform  application  of  reg¬ 
ulations  and  other  sound  procurement 
practices.  The  guidebook  is  also  in¬ 
tended  to  be  helpful  to  Air  Force  manag¬ 
ers  responsible  for  the  procurement  of 
total  TS  or  ATE  systems.  References  to 
appropriate  DOD,  Air  Force,  and  MIL-STD 
documents  are  provided  to  allow  further 
detailed  investigation  into  areas  of 
specific  interest. 

1.2  SCOPE 

This  is  one  of  a  series  of  guidebooks 
related  to  the  Software  Acquisition  Engi¬ 
neering  (SAE)  process  for  TS  and  ATE 
ground  based  systems.  The  SAE  guidebook 
titles  are  listed  below: 

Software  Cost  Measuring  and  Reporting 
Requirements  Specification 
Contracting  for  Software  Acquisition 
Statements  of  Work  (SOW)  and  Requests 
for  Proposal  (RFP) 

Regulations,  Specification  and 
Standards 

Measuring  and  Reporting  Software 
Status 

Computer  Program  Documentation 
Requi rements 


Software  Quality  Assurance 
Verification 

Validation  and  Certification 
Computer  Program  Maintenance 
Configuration  Management 
Reviews  and  Audits 
Management  Reporting  by  the 
Software  Director 

For  the  purposes  of  this  guidebook.  Con¬ 
tracting  for  Software  Acquisition  encom¬ 
passes  solicitation  documents;  proposal 
evaluation;  contractor  selection;  con¬ 
tract  change  management;  and  contract 
compliance.  The  emphasis  is  on  specific 
contracting  considerations,  whereas 
detailed  treatment  of  related  topics 
(e.g. ,  requirement  specification,  RFP, 
SOW,  etc.)  is  referenced  to  other  guide¬ 
books  in  this  series.  Items  that  are 
unique  to  either  TS  or  to  ATE  are  speci¬ 
fically  addressed  in  each  document 
section. 

1.3  TS  AND  ATE  OVERVIEW 

1.3.1  TS  System  Characteristics 

The  TS  system  is  a  combination  of  spe¬ 
cialized  hardware,  computing  equipment, 
and  software  designed  to  provide  a  syn¬ 
thetic  flight  and/or  tactics  environment 
in  which  aircrews  learn,  develop  and 
improve  the  skills  associated  with  indi¬ 
vidual  and  coordinated  tasks  in  specific 
mission  situations.  Visual,  aural,  and/ 
or  motion  systems  may  be  included.  Fig¬ 
ure  1.3-1  depicts  a  typical  training  sim¬ 
ulator  which  employs  digital  processing 
capability. 

The  computer  system,  integral  to  crew 
training  simulator,  can  consist  of  one 
or  more  general  purpose  computers.  The 
computing  hardware  operates  with  float¬ 
ing  point  arithmetic  and  sufficient  bit 
capacity  to  provide  efficient  use  of  a 
simulator  High  Order  Language  (HOL). 

When  a  multi-processor/multi-computer 
system  is  used,  it  must  be  designed  such 


Figure  1.3-1,  Typical  Crew  Training  Simulator 


that  computers  can  operate  simulta¬ 
neously  and  are  controlled/synchronized 
by  a  single  program  (supervisor/  execu¬ 
tive).  The  executive  directs  program  exe¬ 
cution  and  regulates  priorities. 

The  simulator  (1)  accepts  control  inputs 
from  the  trainee  (via  crew  station  con¬ 
trols)  or  from  the  instructor  operator 
station;  (2)  performs  a  real-time  solu¬ 
tion  of  the  simulator  mathematical 
model;  and  (3)  provides  output  responses 
necessary  to  accurately  represent  the 
static  and  dynamic  behavior  of  the  real 
world  system  (within  specified  tolerance 
and  performance  criteria). 

Since  training  simulators  consist  of 
interdependent  hardware  and  software,  a 
joint  hardware/software  development 
effort  is  required.  As  the  complexity  of 
training  simulators  increases,  simula¬ 
tion  software  continues  to  grow  in  com¬ 
plexity,  size,  and  cost.  Software  costs 
can  and  do  exceed  computer  hardware 
costs  in  many  cases.  Therefore,  it  is 
imperative  that  the  simulation  software 
acquisition  engineering  process  be 
subjected  to  formal  system  engineering 
planning  and  discipline  to  ensure  cost- 
effective  procurement. 

1.3.2  ATE  System  Characteristics 

Automatic  Test  Equipment  is  defined  as 
that  ground  support  system  which  per¬ 
forms  vigorous  system  tests  with  minimum 
manual  intervention.  ATE  is  used  in 
place  of  manual  devices  because  it  is 
more  cost  effective,  provides  required 
repeatability,  or  repair  of  the  item 
being  tested  requires  the  speed  which 
only  an  automatic  tester  can  achieve. 

Figure  1.3-2  shows  the  typical  compon¬ 
ents  of  an  ATE  system.  Note  that  there 
are  both  hardware  and  software  elements 
involved.  Most  of  the  elements  shown  in 
the  figure  will  be  found  in  the  majority 


of  ATE  systems  although  the  packaging 
and  interface  design  may  vary  between 
specific  systems. 

The  controls  and  displays  section  con¬ 
sists  of  the  computer  peripheral  devices 
such  as  control  panels,  magnetic  tape 
cassettes  or  disks,  a  cathode  ray  tube 
(CRT),  keyboard,  and  small  printer.  The 
computer  (normally  a  minicomputer),  as 
controlled  by  software,  operates  the 
peripheral  devices;  switches  test  stim¬ 
uli  on  and  off;  and  measures  responses 
of  the  Unit  Under  Test  (UUT)  (comparing 
to  predetermined  values).  The  operator 
maintains  supervisory  control  of  the 
testing  process  through  the  peripherals; 
however,  his  interaction  is  usually  mini¬ 
mal  since,  by  definition,  the  automatic 
test  feature  was  selected  in  preference 
to  an  operator-controlled  test  system. 

ATE  is  normally  designed  to  accommodate 
testing  several  different  articles  of 
system  equipment  (normally  one  at  a 
time).  The  maintenance  level  being  sup¬ 
ported  by  the  ATE  is  determined  by 
logistics  systems  analysis. 

The  importance  of  the  software  portion 
of  the  ATE  system  should  not  be  mini¬ 
mized  since  both  the  application  of  the 
test  stimuli  and  the  measurement  of  the 
result  are  achieved  via  software.  Arbi¬ 
trary  function  generation  and  compli¬ 
cated  wave  analysis  can  also  be 
accomplished  by  software  and  is  becoming 
more  prevalent  in  ATE  systems.  The  cost 
of  ATE  software  is  a  significant  compo¬ 
nent  of  total  ATE  costs  and  design 
trades  can  be  performed  to  minimize  ATE 
life  cycle  costs. 

1.4  GUIDEBOOK  ORGANIZATION 

Section  1.0  of  this  guidebook  contains 
introductory  material  about  the  guide¬ 
book,  including  guidebook  purpose  and 
scope,  and  the  guidebook's  relationship 
to  the  other  SAE  guidebooks.  It  provides 
a  brief  description  of  typical  ATE  and 
TS  systems  and  '  describes  the 

organization  and  use  of  the  guidebook. 
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Figure  1.3-2.  Typical  ATE  Configuration 


Section  2.0  is  a  list  of  key  government 
documents  that  were  referenced  in  the 
preparation  of  this  guidebook. 

Sections  3  through  6  contain  ground  sys¬ 
tems  software  contracting  guidelines. 
Section  3.0  provides  an  overall  software 
contract  perspective  including  factors 
and  trades  that  should  be  considered 
while  still  in  the  pre-contract  phase. 
This  section  is  an  introduction  to  con¬ 
tracting  for  ground  system  software  and 
contains  discussions  specific  to  TS  and 
ATE.  Section  4.0  addresses  the  formal 
steps  involving  the  source  selection 
authority,  and  his  supporting  staff, 
source  lists,  preparation  of  advanced  in¬ 
formation,  and  solicitation  documents. 
It  also  discusses  appropriate  regula¬ 
tions  such  as  AFR70-15.  Section  5.0  dis¬ 
cusses  the  process  of  source  selection 
and  it  includes  provision  of  appropriate 
TS  and  ATE  software  support  to  contrac¬ 
tor  selection.  Section  6.0  covers  those 


areas  pertinent  to  post  contract  award 
including  contract  management,  changes 
to  contracts,  and  contract  compliance. 

Section  7.0  is  a  bibliography  of  docu¬ 
ments  that  are  generally  applicable  to 
the  subject  of  software  contracting. 
This  section  is  an  expansion  of 
Section  2.0,  referenced  documentation. 
The  section  includes  documents  not  spe¬ 
cifically  noted  in  the  text.  Section  8.0 
provides  a  matrix  tabulation  for  the 
cross  reference  relationship  between 
guidebook  topics  and  corresponding  gov¬ 
ernment  documents.  Sections  9,0  and  10.0 
contain  respectively  a  glossary  of  se¬ 
lected  terms  used  in  the  guidebook,  and 
the  expansion  of  a 1 1  abbreviations  and 
acronyms  used  in  the  guidebook.  Section 
11.0  is  a  detailed  subject  index  indicat¬ 
ing  which  guidebook  paragraphs  address 
the  listed  subjects. 


Section  2.0  APPLICABLE  DOCUMENTS 


This  section  lists  documents  which  bear 
on  the  subject  of  contracting  for  soft¬ 
ware  acquisition.  A  contents  summary  of 
principal  documents  is  included  in 
Section  7.0. 

AFR  70-15,  Source  Selection  Pro¬ 
cedures 

AFSC  Supl.,  AFR  70-15,  Source  Selec¬ 
tion  Policy  and  Procedures 

ASD  Supl.,  AFR  70-15,  Source  Selec¬ 
tion  Procedures 

AFR  800-12,  Acquisition  of  Support 
Equipment 

AFR  300-2,  Management  of  the  USAF 
Automatic  Data  Processing  Program 

AFR  800-2,  Acquisition  Program  Manage¬ 
ment 


Armed  Services  Procurement  Regulations 
(ASPR),  Several  of  these  are  applicable 
and  specific  references  are  provided  in 
the  text. 

AFR  310-1,  Management  of  Contractor 
Data 

D0D  5000. 19-L,  Acquisition  Management 
Systems  and  Data  Requirements 
Control  List 

AFR  800-14,  Management  of  Computer  Re¬ 
sources  in  Systems 

AFR  800-6,  Program  Control -Financial 


Section  3.0  SOFTWARE  ACQUISITION 


This  section  summarizes  contracting 
principals  associated  with  ATE  and  TS. 
Particular  emphasis  is  given  to  the  re¬ 
quirement  of  Armed  Services  Procurement 
Regulations  (ASPR)  as  these  apply  to  the 
acquisition  engineering  role.  The  formal 
source  selection  process  is  introduced 
along  with  particular  requirements  of 
AFR70-15  and  its  supplements.  These 
guidelines  are  applicable  to  ATE  and  TS 
hardware  as  well  as  software  since  soft¬ 
ware  is  normally  acquired  as  an  integral 
part  of  the  ATE  or  TS  system.  In  addi¬ 
tion,  guidelines  are  provided  with  empha¬ 
sis  on  those  matters  applicable  to 
software  acquisition  for  ATE  and  TS. 

3.1  ATE  &  TS  SOFTWARE  CONTRACTING 

Contracting  for  ATE  and  TS  software 
acquisition  involves  conformance  with  a 
number  of  DOD,  Air  Force  and  Air  Force 
Systems  Command  regulations  and  other 
forms  of  command  media. 

3.1.1  General  Requirements 

AFR  70-15  establishes  policy  and  pro¬ 
cedures  associated  with  formal  source 
selection.  It  defines  the  source  selec¬ 
tion  organization,  responsibilities  and 
activities  required  to  plan  and  conduct 
source  selection  activities.  It  is  con- 
sistant  with  the  ASPR  which  establish 
fundamental  authority  for  all  procure¬ 
ment  activities  of  the  military 
establishment. 

AFR  70-15  is  applicable  to  major  sys¬ 
tems,  those  requiring  at  least  $50M 
Research  Development  Test  &  Evaluation 
(RDTSE)  funds  or  projected  production 
funds  of  $200M  or  more.  While  few  ATE 
and  TS  programs  normally  require  funding 
at  these  levels,  AFR  70-15,  together 
with  its  AFSC  and  ASD  supplements,  recom¬ 
mend  application  of  its  general  princi¬ 
pals  to  smaller  procurements.  This 
process  is  discussed  in  greater  detail 
in  later  paragraphs. 


3.1.2  Types  of  Contracting 

ASPR  2-102,  2-501,  and  3-200  recognize 
three  primary  types  of  procurement. 

3. 1.2.1  Procurement  by  Formal  Advertis¬ 
ing.  Procurement  by  formal  advertising 
(ASPR  2-102)  involves  preparation  of  de¬ 
tailed  specifications  by  the  Government 
and  making  these  specifications  avail¬ 
able  to  a  preselected  list  of  responsi¬ 
ble  contractors.  Each  contractor  then 
proposes  a  price  in  accordance  with  the 
specifications.  Since  the  government's 
specifications  are  complete,  a  technical 
proposal  from  the  bidders  is  not  neces¬ 
sary.  Award  is  made  to  the  responsible 
bidder  whose  offer  is  most  advantageous 
to  the  government.  While  price  is  the 
fundamental  factor  governing  award, 
other  factors  may  be  considered. 

The  contracts  resulting  from  procurement 
by  formal  advertising  are  of  the  firm 
fixed  price  type,  with  or  without  price 
adjustments  based  on  changing  economic 
factors.  In  such  contracts  the  financial 
liability  to  the  government  is  limited 
by  the  firm  fixed  price  as  adjusted  by 
economic  factor.  The  contractor  bears 
the  fundamental  cost  risk  and  his 
profitability  is  determined  by  cost 
performance.  This  method  of  contracting 
places  the  greatest  emphasis  on  the 
profit  motive.  This  motive  is  the 
fundamental  influence  on  american 
industry. 

Because  of  the  advantages  to  the  govern¬ 
ment  and  the  freedom  on  the  part  of  con¬ 
tractors  to  strive  for  cost  control  as  a 
means  of  maximizing  profits,  ASPR 
2-102.1  requires  this  method  to  be  used 
whenever  such  method  is  feasible  and 
practical. 

It  is  fundamental  to  this  procurement 
concept  that  the  system  being  procured 
can  be  described  via  the  procurement 
specification,  in  such  a  way  that  the 
government  is  assured  of  meeting  its 
total  requirements,  no  matter  which  of 
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several  competing  contractors  is  se¬ 
lected.  Further,  the  specifications 
cannot  in  any  way  favor  any  particular 
contractor's  proprietary  products,  ap¬ 
proaches  or  manufacturing  processes. 
Hence,  the  risk  associated  with  tech¬ 
nical  or  performance  integrity  lies  pri¬ 
marily  with  the  government.  For  this 
reason  fixed  price  contracting  is  rarely 
advantageous  to  the  government  where  fac¬ 
tors  exist  which  preclude  such 
specification. 

The  nature  of  TS  and  ATE  is  such  that  as 
a  matter  of  practicality  procurement  by 
formal  advertising  is  rarely  used.  Train¬ 
ing  simulators  procured  by  the  USAF 
often  involves  advanced  technology  re¬ 
quirements.  The  contractor  often  must 
design  new  hardware  and  software.  Hence, 
a  form  of  contracting  which  places  a 
degree  of  burden  on  the  contractor  to 
ensure  technical  performance  of  his  sys¬ 
tem  is  more  advantageous  to  the  govern¬ 
ment.  The  acquisition  engineer  with  his 
knowledge  of  the  technical  requirements 
of  the  ROC  or  other  originating  material 
and  the  ''state  of  the  art"  of  TS  is  in 
the  best  position  to  assist  in 
determination  of  this  fact. 

Similarly,  ATE  has  not  yet  matured  to 
the  point  at  which  Procurment  by  Formal 
Advertising  is  practicable.  Unique 
adapters,  UUT  test  software,  environ¬ 
mental  considerations  and  other  particu¬ 
lars  affecting  control  software  exist  in 
the  majority  of  USAF  applications.  How¬ 
ever,  as  these  systems  mature  and  become 
more  standardized,  the  time  may  come 
when  USAF  requirements  can  best  be  met 
by  procuring  "off  the  shelf"  ATE.  This 
may  very  well  change  the  method  of  con¬ 
tracting.  The  acquisition  engineer  is  in 
the  best  position  to  know  whether  this 
point  has  been  reached  in  any  particular 
application  and  thus  provide  expert  opin¬ 
ion  during  the  procurement  planning 
phase. 

3. 1.2. 2  Two  Step  Formal  Advertising. 
ASPR  2-501  describes  a  modification  of 
the  method  described  above  which  may 
make  the  method  more  nearly  applicable 


to  TS  and  ATE  procurement.  Called  Two 
Step  Formal  Advertising,  this  method 
expands  the  use  and  obtains  the  benefits 
of  formal  advertising,  where  inadequate 
specification  preclude  the  use  of  con¬ 
ventional  formal  advertising.  It  is  par¬ 
ticularly  useful  in  procurement 
requiring  technical  proposals  for  com¬ 
plex  equipment.  It  is  conducted  in  two 
steps. 

Step  one  consists  of  the  request  for, 
and  submission  evaluation  of,  technical 
proposals  with  pricing.  In  step  one, 
conformity  with  the  government's  tech¬ 
nical  requirements  is  resolved.  During 
this  phase  contractors  whose  technical 
submittals  are  unacceptable,  and  cannot 
be  made  acceptable  without  undue  effort 
both  on  the  part  of  the  government  and 
the  contractor,  can  be  eliminated. 

Step  two  is  then  a  formal  advertised  pro¬ 
curement  which  is  confined  to  those  con¬ 
tractors  who  submitted  acceptable 
technical  proposals  during  step  one. 

Two  Step  Formal  Advertising  Procurements 
result  in  firm  fixed  price  contracts 
with  or  without  economic  price  adjust¬ 
ment.  This  method  combines  the  advantage 
of  maximizing  the  profit  incentive  with 
the  advantage  of  sharing  the  risk  of 
technical  integrity  between  the  govern¬ 
ment  and  the  contractor.  ASPR  2-502 
lists  the  conditions  for  use  of  this 
method  of  procurement.  In  general,  these 
conditions  state  that  available  govern¬ 
ment  specifications  are  not  sufficiently 
definitive  or  complete,  that  definite 
criteria  exists  for  evaluating  technical 
proposals,  that  more  than  one  techni¬ 
cally  qualified  source  is  expected  to  be 
available  and  that  sufficient  time  is 
available  for  use  of  the  two  step 
method. 

The  acquisition  engineer  becomes  more 
deeply  involved  during  the  procurement 
phase  with  Two  Step  than  with  Formal 
Advertising  procurements.  In  the  later 
case  his  principle  involvement  is  in 
preparation  of  the  technical  specifica¬ 
tions  which  accompany  the  solicitation 


documentation.  Guidelines  appropriate  to 
this  activity  are  included  in  paragraph 
4.2. 

In  the  Two  Step  process  the  acquisition 
engineer  is  also  involved  in  evaluation 
of  each  contractor's  technical  and  man¬ 
agement  proposals.  This  evaluation  is 
described  in  paragraphs  5.2  and  5.3. 

3. 1.2.3  Negotiated  Procurements.  The 
majority  of  ATE  and  TS  procurements  are 
performed  under  the  authority  of  19 
U.S.C.  2304(a)  and  described  in 
Section  III  of  the  ASPR  pertaining  to 
Negotiated  Procurements.  Because  of  the 
technical  and  cost  risk  associated  with 
ATE  and  TS  procurements  involving  new 
developmental  hardware  and/or  software, 
it  is  impractical  to  attempt  procurement 
by  the  methods  described  in  paragraphs 
3. 1.2.1  and  3. 1.2. 2,  above.  In  addition, 
ATE  and  TS  are  often  acquired  as  an  inte¬ 
gral  part  of  a  larger  weapon  system  and 
are  procured  by  the  same  acquisition 
method  as  the  larger  system.  Virtually 
all  weapon  systems  acquisitions  are 
negotiated  procurements. 

This  method  differs  from  formal  advertis¬ 
ing  in  that  a  Request  for  Proposal 
(RFP),  containing  the  governments  techni¬ 
cal  requirements,  are  made  available  to 
industry.  Contractors  then  respond  with 
technical,  management  and  cost  propos¬ 
als.  Those  which  are  not  sufficiently 
responsive  are  eliminated  and  contracts 
are  then  definitized  with  the  remaining 
contractors.  Final  prices  associated 
with  these  contracts  are  negotiated  with 
each  responsive  and  responsible  contrac¬ 
tor.  This  negotiation  may  also  involve 
modification  of  the  contractor's  techni¬ 
cal  or  management  proposals.  After  all 
negotiations  are  completed,  an  award  is 
made  to  the  firm  or  firms  whose  negoti¬ 
ated  technical,  management  and  cost  pro¬ 
posals  are  most  advantageous  to  the 
government. 

ASPR  3-200  defines  the  circumstances 
under  which  negotiated  procurement  may 
be  substituted  for  formal  advertising  or 
two  step  procurements.  Of  these,  the 


following  are  applicable  in  the  case  of 
ATE,  TS  and  other  ground  systems. 

a.  ASPR  3-204  constitutes  authority 
for  application  of  negotiated  contracts 
to  personal  or  professional  services  con- 
racts.  Such  contracts  may  be  appropriate 
to  vendor  supplied  software  support  ser¬ 
vices  provided  at  government  facilities. 
This  may  be  required  to  maintain  soft¬ 
ware  for  previously  delivered  ATE  and  TS 
and  to  provide  assistance  in  the  develop¬ 
ment  of  organic  software  maintenance 
capability. 

b.  ASPR  3-211  permits  application  of 
negotiated  contracts  to  experimental,  de¬ 
velopmental  or  research  work.  This  may 
be  applicable  where  the  ATE  or  TS  re¬ 
quires  the  practical  application  of  in¬ 
vestigative  findings  and  theories  of  a 
scientific  nature.  Such  would  be  the 
case  where  the  software  is  both  unique 
and  of  a  scientifically  complex  nature. 
This  category  fits  a  wide  variety  of  TS 
software  and  algorithms.  It  should  be 
noted,  however,  that  the  authority  estab¬ 
lished  by  ASPR  3-211  does  not  apply  to 
negotiation  of  contracts  involving  quan¬ 
tity  production.  Hence,  while  it  may  be 
applicable  to  prototype  development  of 
ATE  and  TS,  particularly  the  software, 
it  would  not  normally  apply  to  quantity 
production  versions  of  these  systems. 

c.  ASPR  3-212  establishes  the  author¬ 
ity  for  procurement  by  negotiation  where 
the  ATE  or  TS  is  classified  "confiden¬ 
tial"  or  higher  and  this  classification 
would  preclude  release  to  the  public  of 
the  necessary  solicitation  documents. 

d.  ASPR  3-214  provides  the  authority 
which  Is  applicable  to  most  aerospace 
systems  where  negotiated  procurement  is 
authorized.  It  indicates  negotiated 
procurement  is  authorized.  It  indicates 
negotiated  procurements  may  be  used 
where  it  is  determined  that  the  system 
requires  substantial  initial  investment 
or  an  extended  period  of  preparation  for 
manufacture.  This  is  certainly  applica¬ 
ble  to  ATE  or  TS  systems  normally  pro¬ 
cured  by  the  USAF.  ASPR  3-214  further 


indicates  its  authority  may  be  used  for 
the  procurement  of  "technical  or  special¬ 
ized  suppl ies--for  example:  aircraft, 
tanks,  radar,  guided  missiles,  rockets, 
and  similar  items  of  equipment;  major 
components  of  any  of  the  foregoing;  and 
any  supplies  of  a  technical  or  special¬ 
ized  nature  which  may  be  necessary  for 
the  use  or  operation  of  any  of  the  fore¬ 
going."  The  majority  of  ATE  and  TS 
procured  by  the  USAF  fits  this  category. 

e.  ASPR  3-215  permits  negotiation 
where  previous  attempts  to  procure  the 
system  by  formal  advertising  have  failed 
because  only  unreasonable  bids  were 
recei ved. 

(1)  Industry  normally  objects  to 
the  firm  fixed  price  contracts  resulting 
from  advertised  procurements  where  the 
system  involves  new  or  modified  hardware 
or  software.  Because  most  USAF  procure¬ 
ments,  ATE  and  TS  included,  are  highly 
competitive  it  is  difficult  for  the  con¬ 
tractor  to  add  significant  markup,  or 
gross  profit,  to  his  estimated  costs  and 
still  produce  a  competitive  price.  Yet, 
this  markup  is  the  contractor's  only  con¬ 
tingency  to  offset  unexpected  costs 
which  nearly  always  arise  during  the 
development  of  hardware  and  software.  It 
may  therefore  be  frequently  impractical 
to  attract  responsible  bidders  for  com¬ 
plex  ATE  and  TS  if  the  firm  fixed  price 
contract  is  to  be  used.  This  will  then 
preclude  advertised  procurements  and  the 
authority  provided  by  ASPR  3-215  may 
therefore  apply. 

(2)  ASPR  3-402  recognizes  a  num¬ 
ber  of  contract  types  which  may  be  used 
with  negotiated  procurements.  Those 
which  are  normally  used  for  ATE  and  TS 
negotiated  procurements  are  summarized 
here.  While  the  final  decision  of  which 
type  of  contract  to  be  used  rarely  is 
made  by  the  acquisition  engineer,  never¬ 
theless  his  recommendations  in  this 
regard,  are  of  significant  importance. 
The  determination  of  contract  type  is 
largely  based  on  the  technical  content 
of  the  ATE  or  TS  system  to  be  acquired. 
The  acquisition  engineer's  information 


is  fundamental  to  this  process.  Sec¬ 
ondly,  the  type  of  contracting  method  is 
of  extreme  significance  in  the  number 
and  quality  of  contractors  attracted  by 
the  procurement.  This  in  turn  affects 
the  amount  of  post  award  government  ef¬ 
fort  required  to  properly  manage  the 
contractor.  Hence,  it  is  particularly 
important  that  the  acquisition  engineer 
influence  this  selection  process. 

(a)  Cost-Reimbursement  Type 
Contracts  (ASPR  3-405) 

The  cost-reimbursement  type  of  contract 
provides  for  payment  to  the  contractor 
of  allowable  costs  incurred  in  the  per¬ 
formance  of  the  contract  to  the  extent 
prescribed  in  the  contract.  This  type  of 
contract  establishes  an  estimate  of 
total  cost  for  the  purpose  of  (1)  obliga¬ 
tion  of  funds,  and  (2)  establishing  a 
ceiling  which  the  contractor  may  not 
exceed  (except  at  his  own  risk)  without 
prior  approval  and  subsequent  ratifica¬ 
tion  by  the  contracting  officer. 

The  cost-reimbursement  type  contract  is 
suitable  for  use  only  when  the  uncertain¬ 
ties  involved  in  contract  performance 
are  of  such  magnitude  that  cost  of  per¬ 
formance  cannot  be  estimated  with  suffi¬ 
cient  reasonableness  to  permit  use  of 
any  type  of  fixed-price  contract.  In  ad¬ 
dition,  it  is  essential  that  (1)  the  con¬ 
tractor's  cost  accounting  system  is 
adequate  for  the  determination  of  costs 
applicable  to  the  contract  and  (2)  ap- 
priate  surveillance  by  government  person¬ 
nel  during  performance  will  give 
reasonable  assurance  that  inefficient  or 
wasteful  methods  are  not  being  used. 

Major  weapon  systems  are  normally 
procured  by  this  method.  Supplemental 
agreements  to  the  prime  weapon  or  air¬ 
craft  system  contract  for  the  provision 
of  ATE  or  TS  are  therefore  of  this  type. 
In  addition  this  type  is  useful  where 
advanced  technology  hardware  of  software 
requirements  exist  for  TS  and  ATE  regard¬ 
less  of  whether  they  are  procured  as  an 
integral  part  of  a  larger  system  or 
procured  on  a  stand  alone  basis. 


(b)  Cost-Plus-Award-Fee  (CPAF) 
Contract  (ASPR  3-405.5) 

The  CPAF  contract  is  a  cost  reimburse¬ 
ment  type  of  contract  with  special  fee 
provisions.  It  provides  a  means  of  apply¬ 
ing  incentives  in  contracts  which  are 
not  susceptible  to  finite  measurements 
of  performance  necessary  for  structuring 
incentive  contracts.  The  fee  established 
in  a  CPAF  contract  consists  of  two 
parts:  (1)  a  fixed  amount  which  does  not 
vary  with  performance,  and  (2)  an  award 
amount,  in  addition  to  the  fixed  amount, 
sufficient  to  provide  motivation  for 
excellence  in  contract  performance  in 
areas  such  as  quality,  timeliness, 
ingenuity,  and  cost  effectiveness.  Award 
fee  may  be  earned  by  the  contractor  in 
whole  or  in  part.  The  amount  of  award 
fee  to  be  paid  is  based  on  evaluation  by 
the  government  of  the  quality  of  the  con¬ 
tractor's  performance,  judged  in  the 
light  of  criteria  set  forth  in  the  con¬ 
tract.  The  number  of  criteria  used  and 
the  requirements  which  are  represented 
will  differ  widely  from  one  contract  to 
another.  Therefore,  when  determining 
criteria  and  rating  plans,  the  using  ac¬ 
tivity  should  be  flexible  and  select  a 
plan  which  will  motivate  the  contractor 
in  a  positive  way  to  improve  perfor¬ 
mance.  Evaluations  are  furnished  to  the 
contractor  to  afford  him  an  opportunity 
to  comment  on  the  evaluation  findings. 
The  decision  that  an  award  fee  has  been 
earned  is  based  on  the  reports  of  perfor¬ 
mance  made  by  the  government  personnel 
knowledgeable  with  respect  to  the  con¬ 
tract  requirements.  The  ATE  or  TS  engin¬ 
eers  role  is  of  primary  importance  in 
this  process  and  requires  detailed  know¬ 
ledge  of  the  system  requirements  and  of 
the  contractor's  performance  against 
those  requirements.  The  former  is  cov¬ 
ered  in  the  Requirements  Specification 
Guidebook,  CDRL  A004. 

(c)  Cost-Plus- Incentive-Fee 
(CPIF)  Contract  (ASPR  3-405.4) 

The  cost-plus-incentive-fee  contract  is 
a  cost-reimbursement  type  contract  with 
provision  for  a  fee  which  is  adjusted  by 


a  formula  in  accordance  with  the  rela¬ 
tionship  total  allowable  costs  bear  to 
target  cost.  Under  this  type  of  con¬ 
tract,  there  is  negotiated  initially  a 
target  cost,  a  target  fee,  a  minimum  and 
maximum  fee,  and  a  fee  adjustment  formu¬ 
la.  After  performance  of  the  contract, 
the  fee  payable  to  the  contractor  is  de¬ 
termined  in  accordance  with  the  formula. 
The  formula  provides,  within  limits,  for 
increases  in  fee  above  target  fee  when 
total  allowable  costs  are  less  than  tar¬ 
get  costs,  and  decreases  in  fee  below 
target  fee  when  total  allowable  costs 
exceed  target  costs.  The  provision  for 
increase  or  decrease  in  the  fee  is 
designed  to  provide  an  incentive  for 
maximum  effort  on  the  part  of  the  con¬ 
tractor  to  manage  the  contract 
effectively. 

The  cost-plus-incentive-fee  contract  is 
suitable  for  use  primarily  for  develop¬ 
ment  and  test  when  a  cost-reimbursement 
type  of  contract  is  found  necessary  in 
accordance  with  ASPR  3-405  and  when  a 
target  and  a  fee  adjustment  formula  can 
be  negotiated  which  are  likely  to  pro¬ 
vide  the  contractor  with  a  positive  pro¬ 
fit  incentive  for  effective  management. 

(d)  Cost-Plus-Fixed-Fee  (CPFF) 
Contract  (ASPR  3-405.6) 

The  cost-plus-fixed-fee  contract  is  a 
cost  reimbursement  type  of  contract 
which  provides  for  the  payment  of  a 
fixed  fee  to  the  contractor.  The  fixed 
fee  once  negotiated  does  not  vary  with 
actual  cost,  but  may  be  adjusted  as  a 
result  of  any  subsequent  changes  in  the 
work  or  services  to  be  performed  under 
the  contract.  Because  the  fixed  fee  does 
not  vary  in  relation  to  the  contractor's 
ability  to  control  costs,  the  cost-plus- 
fixed-fee  contract  provides  the  contrac¬ 
tor  with  only  a  minimum  incentive  for 
effective  management  control  of  costs. 

The  cost-plus-fixed-fee  contract  is  suit¬ 
able  for  use  when: 

1.  a  cost  reimbursement  type 
of  contract  Ts  found  necessary  in  accord¬ 
ance  with  3-405; 
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2.  the  parties  agree  that 

the  contract  should  be  fee  bearing; 

2-  The  contract  is  for  the 
performance  of  research,  or  preliminary 
exploration  or  study,  where  the  level  of 
effort  required  is  unknown;  and  where 

measuring  achievements  in  contract  per¬ 
formance  does  not  lend  itself  to  the  sub¬ 
jective  evaluation  required  in  CPAF 

contracts;  or 

4.  the  contract  is  for  devel¬ 
opment  of  ATT  or  TS  once  preliminary  ex¬ 
ploration,  studies,  and  risk  reduction 

have  indicated  a  high  degree  of  probabil¬ 
ity  that  the  development  is  achievable 
and  the  government  generally  has  deter¬ 
mined  its  desired  performance  objective 
and  schedule  of  completion.  ASPR  3-405.6 
precludes  or  limits  the  use  of  this  con¬ 
tracting  method  in  a  number  of  appli¬ 
cable  situations.  Consequently  the  use 
of  CPFF  as  opposed  to  CPAF  or  one  of  the 
other  methods  previously  indicated 
should  be  considered.  It  may  be  noted, 
however,  that  CPFF  was  the  method  se¬ 
lected  for  development  of  these  Software 
Acquisition  Engineering  Guidebooks. 

(e)  Fixed-Price  Incentive  Con¬ 
tracts  (ASPR  3-404.4) 

The  fixed-price  incentive  contract  is  a 
fixed-price  type  contract,  with  provi¬ 
sion  for  adjustment  of  profit  and  estab¬ 
lishment  of  the  final  contract  price  by 
a  formula,  based  on  the  relationship 
final  negotiated  total  cost  bears  to 
total  target  costs. 

Under  this  type  of  incentive  contract, 
there  is  negotiated  at  the  outset,  a  tar¬ 
get  cost,  a  target  profit,  a  price  ceil¬ 
ing  (but  not  a  profit  ceiling  or  floor), 
and  a  formula  for  establishing  final  pro¬ 
fit  and  price.  After  performance  of  the 
contract,  the  final  cost  is  negotiated 
and  the  final  contract  price  is  then 
established  in  accordance  with  the  for¬ 
mula.  When  the  final  cost  is  less  than 
target  cost,  application  of  the  formula 
results  in  a  final  profit  greater  than 
the  target  profit,  conversely,  when 


final  cost  is  more  than  target  cost, 
application  of  the  formula  results  in  a 
final  profit  less  than  the  target  pro¬ 
fit,  or  even  a  net  loss.  Thus,  within 
the  price  ceiling,  the  formula  provides 
for  the  government  and  the  contractor  to 
share  the  responsibility  for  costs 
greater  or  less  than  those  originally 
estimated,  as  determined  by  a  comparison 
of  negotiated  final  cost  with  target 
cost.  Because  the  profit  resulting  from 
application  of  the  formula  is  in  inverse 
relationship  to  costs,  the  formula  pro¬ 
vides  the  contractor  in  advance  with  a 
calculable  profit  incentive  to  control 
costs.  This  contracting  method  has  been 
used  for  TS  and  ATE  where  the  system  is 
technically  complex  and  advanced  from 
the  standpoint  of  the  total  ATE  or  TS 
system  view  point.  It  is  the  method  nor¬ 
mally  used  for  procurement  of  opera¬ 
tional  flight  training  simulators  by  the 
USAF.  But  in  these  cases,  each  major  ele¬ 
ment  of  hardware  and  software  involves 
only  technology  which  is  well  within  the 
state  of  the  art  and,  in  most  cases,  has 
been  demonstrated  on  previously  deliv¬ 
ered  ATE  and  TS. 

(f)  Time  and  Materials  Con¬ 
tracts  (ASPR  3-406.1) 

The  time  and  materials  type  of  contract 
provides  for  the  procurement  of  supplies 
or  services  on  the  basis  of  (1)  direct 
labor  hours  at  specified  fixed  hourly 
rates  and  (2)  material  at  cost. 

This  contract  type  is  not  applicable  to 
ATE  and  TS  procurement  generally.  How¬ 
ever,  it  may  apply  to  contractor  post 
delivery  support  which  may  be  required 
to  maintain  ATE  or  TS  software  and  to 
assist  the  USAF  in  development  and  main¬ 
tenance  of  organic  software  or  hardware 
maintenance  capability. 

(g)  Additional  Incentives  (ASPR 

3-407) 

In  addition  to  the  profit  incentives  to 
control  costs  inherent  in  many  of  the 
contract  types  and  combinations  thereof 
described  above,  there  are  other  means 


14 


of  providing  profit  incentives  to  con¬ 
tractors  to  obtain  extra  management  at¬ 
tention  and  effort.  Increases  in  profits 
or  fees  resulting  from  use  of  incentive 
provisions  may  be  applied  where  cost, 
performance,  or  other  contractual  goals 
or  standards  have  been  surpassed. 

It  is  important  for  the  ATE  and  TS  ac¬ 
quisition  engineer  to  understand  these 
concepts.  Incentives  are  awarded  on  the 
basis  of  the  government's  evaluation  of 
contractor  performance.  Further,  when 
such  incentives  are  used,  senior  contrac¬ 
tor  management  performance  is  frequently 
evaluated  by  their  company  officials  on 
the  basis  of  the  degree  to  which  the  con¬ 
tractor  is  successful  in  earning  these 
additional  incentives.  Often  incentives 
are  awarded  on  the  basis  of  system  per¬ 
formance.  Contractors  place  emphasis  on 
convincing  the  government  their  perform¬ 
ance  warrants  maximum  incentive  award. 
Therefore,  the  acquisition  engineer  is  a 
primary  factor,  representing  the  govern¬ 
ment's  interests  in  such  evaluations. 

The  acquisition  engineer,  to  represent 
these  interests,  requires  indepth  under¬ 
standing  of  the  ATE  or  TS  system  require¬ 
ments.  It  also  requires  that  the 
acquisition  engineer  have  detailed  under¬ 
standing  of  the  contractor's  actual  per¬ 
formance,  including  performance  of  the 
ATE  or  TS  hardware  and  software,  against 
these  requirements. 

A  contract  with  a  performance  incentive 
is  one  which  incorporates  an  incentive 
to  the  contractor  to  surpass  stated  per¬ 
formance  targets.  The  incentive  provides 
for  increases  in  the  fee  to  the  extent 
that  such  targets  are  surpassed  and  for 
decreases  to  the  extent  that  such 
targets  are  not  met. 

"Performance"  refers  not  only  to  the 
performance  of  the  article  being  pro¬ 
cured,  but  to  the  performance  of  the  con¬ 
tractor  as  well.  Performance  which  is 
the  minimum  the  government  will  accept, 
warrants  only  the  minimum  profit  or  fee. 


Performance  which  meets  the  stated  tar¬ 
gets  will  warrant  the  "target"  profit  or 
fee.  Performance  which  surpasses  these 
targets  are  rewarded  by  additional  prof¬ 
it  or  fee.  The  incentive  feature  is 
applied  to  performance  targets  rather 
than  minimum  performance  requirements. 

The  incentive  should  relate  to  specific 
performance  characteristics.  High  over¬ 
all  performance  is  the  primary  objective 
of  such  contracts.  When  applied  to  the 
performance  of  the  contractor,  the 
incentive  should  relate  to  specific  per¬ 
formance  areas  or  milestones,  such  as 
delivery  or  test  schedules,  quality  con¬ 
trols,  maintenance  requirements,  soft¬ 
ware  reliability,  adequacy  and  timelines 
of  software  documentation,  suitability 
of  algorithms  used  by  the  contractor, 
etc. 

3. 1.2.4  Four-Step  Procurement.  In  1976 
the  Deputy  Secretary  of  Defense  issued  a 
revised  DOD  Directive  4105.62  (Selection 
of  Contractual  Sources  for  Major  Defense 
Systems)  to  update  the  policies  and  pro¬ 
cedures  applicable  to  source  selection 
of  major  weapon  systems.  Included  in 
this  revised  directive  was  the  establish¬ 
ment  of  a  service  test  for  a  proposed 
Four-Step  selection  process  for  research 
and  development  procurements  of  advanced 
engineering,  and  operational  systems. 
Figure  3.1-1  illustrates  this  process. 

As  of  this  writing  the  process  is  still 
under  test.  Depending  upon  the  outcome 
of  the  tests,  Four-Step  procurement  may 
be  adopted  and  would  therefore  be  appli¬ 
cable  to  ATE  and  TS  procurement.  Such 
systems  have  been  identified  for  procure¬ 
ment  by  the  Four-Step  process  during  the 
test  period.  The  objectives  of  the  pro¬ 
posed  Four-Step  process  are  to  improve 
the  overall  source  selection  process,  to 
eliminate  or  reduce  the  opportunity  for 
government  technical  leveling  and  auc¬ 
tioning,  and  industry  "buy-ins."  The 
Four-Steps  in  this  process  are  (1)  Sub¬ 
mission  and  Evaluation  of  the  Technical 
Proposal,  (2)  Submission  and  Evaluation 
of  the  Cost  Proposal,  (3)  Selection  of 
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Apparent  Winning  Contractor,  and  (4)  Ne¬ 
gotiation  and  Award  of  Definitive 
Contract. 

This  process  differs  from  the  conven¬ 
tional  mode  of  procurement  in  two  ways: 

a.  The  Four-Step  process  requires  sep¬ 
arate  submission  of  the  technical  and 
cost  proposal  whereas  in  the  conven¬ 
tional  method,  technical  and  cost 
proposals  are  submitted  simultaneously. 
Thus  all  contractor's  technical  propos¬ 
als  are  evaluated  separately  from  cost 
proposals. 

b.  In  the  Four-Step  process,  one  con¬ 
tractor,  of  those  in  the  competitive 
range,  is  selected  for  negotiation  of  a 
definitive  contract.  In  the  conventional 
method,  definitive  contracts  are  negoti¬ 
ated  with  all  contractors  in  the  competi¬ 
tive  range.  This  would  have  the  effect 
of  reducing  the  time  and  effort  spent  in 
contract  negotiation. 

3.2  PROCUREMENT  PLANNING 


AFSC  Supplement,  AFR  70-15  discourages 
use  of  formal  source  selection  proce¬ 
dures  for  systems  whose  projected  costs 
are  less  than  $1  million.  (Most  ATE  and 
TS  programs  exceed  this  value)  Rather  it 
suggests  using  award  procedures  speci¬ 
fied  in  ASPR  3-805  and  4-106. 

3.2.2  Source  Selection  Organization 

The  initial  process  by  which  the  pro¬ 
curement  phase  is  initiated  is  the  prepa¬ 
ration  and  approval  of  required 
operational  capability  and  the  establish¬ 
ment  or  delegation  of  the  source  selec¬ 
tion  organization. 

3. 2. 2.1  Source  Selection  Organization. 
The  primary  organization  created  for  the 
purpose  of  source  selection  consists  of 
the  Source  Selection  Authority  (SSA)  and 
those  individuals  appointed  to  advise 
and  support  his  source  selection  activi¬ 
ties.  Advisors  normally  consist  of  the 
Source  Selection  Advisory  Council  (SSAC) 
and  the  Source  Selection  Evaluation 
Board  (SSEB). 


The  processes  by  which  source  selection 
is  performed  in  accordance  with  applica¬ 
ble  regulations  and  other  command  media 
generally  consists  of  four  phases.  These 
phases  consist  of  organizing  for  and 
planning  the  source  selection;  prepara¬ 
tion  and  release  of  information  to  re¬ 
sponsible  bidding  contractors;  receipt 
and  evaluation  of  contractor's  propos¬ 
als:  and  selection  and  notification  of 
contract  awards.  The  first  of  these 
phases  is  described  in  this  section. 
Preparation  and  release  of  information 
is  described  in  Section  4  and  Section  5 
describes  the  remaining  phases. 

3.2.1  Formal  Source  Selection  Process 

Figure  3.2-1  provides  an  events  overview 
of  formal  source  selection  processes  as 
contained  in  AFR  70-15  and  its  supple¬ 
ments.  A  more  detailed  illustration  of 
these  activities  is  provided  in 
Figure  5.1-1. 


a.  AFR  70-15  defines  the  SSA  as  the 
offical  designated  to  direct  the  source 
selection  process,  approve  the  selection 
plan,  select  the  source(s)  and  announce 
the  contract  award.  The  Secretary  of  the 
Air  Force;  undersecretaries  or  assistant 
secretaries;  or  chief  of  staff  are  nor¬ 
mally  designated  SSA  for  major  procure¬ 
ments.  However,  in  the  case  of  most  ATE 
or  TS  procurements  this  authority  is 
delegated  to  Hq.  Aeronautical  Systems 
Division.  The  authority  usually  resides 
with  a  senior  staff  officer  such  as  Dep¬ 
uty  for  Systems  (SD),  Deputy  for  Aeronau¬ 
tical  Equipment  (AE ) ,  Deputy  for 
Engineering  (EN) ,  etc. 

b.  AFR  70-15  recognizes  both  the  SSAC 
and  the  SSEB  as  advisory  boards,  support¬ 
ing  the  SSA  during  procurement.  The  for¬ 
mer  is  a  group  of  senior  military  and/or 
government  civilian  personnel.  The  lat¬ 
ter  is  a  group  of  military  and/or  govern¬ 
ment  civilian  personnel,  representing 
various  functional  and  technical  disci¬ 
plines.  The  SSEB  controls  and  evaluates 


vendor  proposals  and  develops  summary 
findings  for  recommendation  to  the  SSA 
through  the  SSAC.  Normally  the  program 
manager,  that  person  responsible  for 
management  of  a  program  in  accordance 
with  AFR  800-2,  is  chairman  of  the  SSEB. 
Other  acquisition  engineers  make  up  the 
bocty  of  the  SSEB  organization. 

c.  For  most  ATE  and  TS  procurements 
it  is  unnecessary  to  use  an  organization 
with  this  degree  of  formality.  Normally 
the  functions  of  the  SSAC  and  SSEB  are 
combined  in  one  organization  and  is 
identified  as  the  Source  Selection  Eval¬ 
uation  Committee  (SSEC).  Figure  3.2-2 
describes  a  typical  organization  appro¬ 
priate  to  procurement  of  TS,  ATE  and 
other  ground  systems.  Included  in  the 
organization  are  those  acquisition  engi¬ 
neering  skills  necessary  to  prepare  the 
source  selection  plan,  select  prospec¬ 
tive  contractor  sources,  prepare  the 
necessary  solicitation  documents,  eval¬ 
uate  proposals  and  make  source  selection 
recommendations  to  the  SSA. 

d.  Particular  requirements  of  the  pro¬ 
curement  will  normally  dictate  the  need 
for  particular  skills.  For  example  a  TS 
with  complex  visual  system  requirements 
may  dictate  need  for  computer  generated 
imagery  and/or  camera  model  specialists 
in  the  SSEC  organization.  An  ATE  procure¬ 
ment  may  be  such  that  ATLAS  language  spe¬ 
cialists  be  included  either  as  consul¬ 
tants  to  the  SSEC  or  as  part  of  the  SSEC 
acquisition  engineering  staff.  In  gen¬ 
eral,  however,  the  primary  organiza¬ 
tional  elements  shown  in  Figure  3.2-2 
should  be  staffed  with  ATE  or  TS  acquisi¬ 
tion  engineers  with  board  knowledge  of 
state  of  the  art  of  ATE/TS  software,  in¬ 
cluding  recent  new  developments.  They 
should  be  familiar  with  the  primary  USAF 
suppliers  of  such  systems,  and  should  be 
well  informed  on  the  using  command's  re¬ 
quirements  for  the  ATE  or  TS  being 
procured. 

3. 2. 2. 2  Responsibilities.  AFR  70-15 
defines  SSA  responsibilities.  Generally 
the  SSA  is  responsible  for  the  proper 
conduct  of  the  source  selection  process. 


He  approves  the  source  selection  plan. 
The  SSA  appoints  the  SSEC  chairman  and 
approves  selection  of  SSEC  members.  He 
must,  through  control  of  disclosure  of 
sensitive  information  and  other  actions, 
ensure  the  procurement  is  made  in  a  fair 
and  impartial  manner.  He  approves  the  so¬ 
licitation  documentation.  The  SSA  estab¬ 
lishes  criteria  for  the  evaluation  of 
contractor  proposals  and  approves  the 
determination  of  those  contractor's 
whose  proposals  have  been  determined  by 
the  SSEC  and  Contracting  Officer  to  be 
in  the  competitive  range.  He  approves 
the  final  contractor  selection  and  pro¬ 
vides  information  as  necessary  to  higher 
authority  concerning  all  matters  of  the 
source  selection  process.  Responsibili¬ 
ties  of  the  SSEC  are  to  perform  those 
activities  necessary  to  support  the  SSA. 
An  overview  of  these  activities  is  illus¬ 
trated  in  Figure  3.2-1. 

3.2.3  Source  Selection  Plan 

AFR  70-15  and  its  supplements,  the  ASPR 
and  other  command  media  require  prepara¬ 
tion  of  plan  for  approval  of  the  SSA, 
for  organizing  and  conducting  proposal 
analysis  and  source  selection.  Prepara¬ 
tion  of  this  plan  is  the  initial  sig¬ 
nificant  task  of  the  SSEC.  Software 
acquisition  engineers  assigned  to  the 
SSEC  play  a  significant  role  in  prepa¬ 
ration  of  •  the  plan.  The  primary  factors 
to  be  considered  are: 

a.  The  plan  describes  the  time  phased 
processes  by  which  potential  contractors 
are  to  be  screened.  ASPR  1-902  directs 
that  contract  awards  shall  be  made  to 
only  qualified  and  responsible  bidders. 
In  addition,  ASPR  1.903  and  3-802.2 
prescribe  the  establishment  and  mainte¬ 
nance  of  qualified  bidders'  lists.  The 
software  acquisition  engineer  must  deter¬ 
mine  whether  potential  contractors  have 
the  technical,  management,  financial, 
manufacturing  and  appropriate  facility 
capability.  Screening  criteria  including 
pre-bidder  review  meetings,  plant  vis¬ 
its,  etc.,  necessary  to  conduct  this 
screening  should  be  included  in  the 
plan. 
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b.  The  procurement  plan  should  con¬ 
tain  a  description  of  criteria  and  tech¬ 
niques  to  be  used  for  evaluation  of 
contractor's  proposals.  One  such 
criteria  is  the  degree  to  which  the  con¬ 
tractor's  proposal  reflecting  a  proposed 
ATE  or  TS,  meets  requirements  of  the  RFP 
and  the  Required  Operational  Capability 
(ROC).  Another  criteria  is  the  measure 
of  risk,  that  the  contractor  can  perform 
the  contract  in  accordance  with  the  re¬ 
quirements  of  the  RFP  and  of  his  pro¬ 
posal.  The  plan  should  include  important 
performance  criteria  against  which  con¬ 
tractors'  proposals  are  to  be  evaluated. 
Examples  of  such  criteria  include  soft¬ 
ware  algorithm  accuracy  (e.g. ,  alititude 
must  be  simulated  to  an  accuracy  of  1 
foot),  and  language  requirements  (e.g., 
the  ATLAS  language  must  conform  to  IEEE 
standard  416),  etc. 

c.  The  source  selection  organization 
should  be  defined  in  the  plan.  A  more  de¬ 
tailed  breakdown  of  the  organizational 
entities  reflected  in  Figure  3.2-2 
should  be  included  with  emphasis  on 
those  particular  skills  and  resources 
needed  within  the  procurement  organiza¬ 
tion.  Further  it  should  reflect  any 
requested  delegation  of  authority  includ¬ 
ing,  where  appropriate,  delegation  of 
the  SSA  to  a  lower  level  organization. 

d.  The  plan  should  describe  those  ele¬ 
ments  of  the  TS  or  ATE  for  which  costs 
will  be  evaluated.  Of  course  total  cost 
is  always  an  evaluation  criteria;  how¬ 
ever,  it  may  also  be  desirable  to  evalu¬ 
ate  the  contractor's  proposed  software 
costs.  If  so,  it  is  necessary  to  include 
in  the  plan  a  requirement  to  ensure  that 
contractors  propose  separately  identifi¬ 
able  software  costs.  In  addition,  the 
plan  should  indicate  the  steps  by  which 
the  government's  independent  software 
cost  estimate  (ICE)  will  be  determined. 
Provided  the  ICE  is  a  reasonable  esti¬ 
mate  of  true  costs  for  the  ATE  or  TS  to 
be  procured,  comparison  of  this  to  the 
contractors'  proposed  software  costs  pro¬ 
vides  a  powerful  tool  for  evaluating 
whether  the  contractor  really  under¬ 
stands  the  software  requirement  for  the 
ATE  or  TS  he  is  proposing. 


e.  Other  items  for  inclusion  in  the 
source  selection  plan  are  required  by 
the  applicable  documentation  listed  in 
Section  2.0. 

3.3  SPECIAL  REQUIREMENTS  FOR  SOFTWARE 
ACQUISITION 

Software  is  normally  a  high  risk  element 
in  TS  and  ATE  procurement.  Where  contrac¬ 
tors  fail  to  meet  cost,  schedule  and 
technical  requirements,  the  difficulty 
can  often  be  traced  to  a  lack  of  soft¬ 
ware  capability.  This  section  describes 
guidelines  which  may  be  incorporated 
into  contracting  for  software  acquisi¬ 
tion  as  a  means  of  reducing  this  risk. 

a.  As  previously  indicated,  a  primary 
requirement  of  the  ASPR  is  that  contrac¬ 
tors  will  be  solicited  from  only  quali¬ 
fied  sources.  This  process  is  summarized 
in  paragraph  5.1.3.  One  method  which  may 
be  used  to  eliminate  unqualified  sources 
is  the  imposition  of  a  minimum  applica¬ 
ble  experience  criteria.  Such  criteria 
might,  for  example,  state  that  contrac¬ 
tors  will  be  accepted  only  from  those 
contractors  who  have  previously  deliv¬ 
ered  a  TS  or  ATE  system  meeting  speci¬ 
fied  requirements  to  a  branch  of  the 
U.S.  military  establishment.  Specific 
software  requirements  should  be  included 
with  the  criteria.  If  such  criteria  is 
to  be  placed  in  the  RFP  it  should  be  de¬ 
scribed  in  the  Source  Selection  Plan  and 
receive  prior  approval  of  the  SSA. 

b.  Particular  attention  should  be 
given  to  definition  of  those  items  of 
software  and  associated  documentation  to 
be  delivered  with  the  ATE  or  TS.  Most 
aerospace  ground  equipment  (AGE),  includ¬ 
ing  ATE  and  TS,  are  procured  in  a  form 
and  quality  generally  described  as  "best 
commercial  practice."  This  means  that 
rather  than  requirements  being  defined 
by  applicable  Mil  Spec,  normal  commer¬ 
cial  practices  are  satisfactory.  This 
can  have  significant  impact  to  the  USAF 
when  procuring  A1l  or  TS  software.  It 
may  mean,  for  example,  that  no  source 
program  listings  will  be  provided  unless 
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specifically  required  by  the  RFP.  Docu¬ 
mentation  for  the  computer  system, 
ATLAS,  FORTRAN,  BASIC,  loading  routines, 
data  base  management  software,  etc. ,  may 
be  limited  to  broad  descriptive  bro¬ 
chures  unless  precise  requirements  are 
contained  in  the  RFP.  It  is  particularly 
important  to  completely  specify  software 
form  and  content  and  deliverable  documen¬ 
tation  requirements  where  it  is  intended 
that  the  TS  or  ATE  system  is  to  be  organ¬ 
ically  maintained  by  the  using  command. 
In  most  cases  the  RFP  should  specifi¬ 
cally  require  delivery  of  source 
listings  as  well  as  object  code. 

c.  Care  should  be  exercised  in  the  se¬ 
lection  of  contractor  teams.  It  is  com¬ 
mon  that  the  prime  contractor  will 
obtain  components  from  subcontractors. 
ATE  suppliers  purchase  computers  and  com¬ 
puter  peripherals,  power  supplies,  fre- 
uency  counters,  and  other  stimuli i  and 
measurement  devices  from  commercial  sup¬ 
pliers  of  such  items.  Similarly  a  TS 
prime  contractor  purchases  computers  and 
instruments  from  subcontractors  and  may 
also  subcontract  a  significant  portion 
of  the  job,  such  as  the  visual  system. 
Because  the  majority  of  contractor  fur¬ 
nished  software  for  these  systems  is  de¬ 
veloped  uniquely  for  each  application, 
it  is  an  integral  part  of  the  hardware 
system.  This  fact  makes  it  more  diffi¬ 
cult  to  subcontract  certain  portions  of 
the  software.  This  is  particularly  true 
of  TS  software  and  UUT  Test  software  as 
well  as  ATE  system  unique  control  and 
support  software  such  as  instrument 
"templates"  and  drivers.  Therefore  par¬ 
ticular  emphasis  should  be  placed  by  the 
acquisition  engineer  in  ensuring  that 
any  proposed  subcontract  relationships 
are  well  defined,  that  interfaces  be¬ 
tween  the  prime  and  subcontractors  are 
well  understood  and  that  adequate  means 
will  be  employed  by  the  contractor  to  en¬ 
sure  technical  integrity  of  the  total 
software  system.  In  addition  it  may  be 
useful  to  reserve  the  right  for  USAF 
prior  approval  of  subcontracts  and 
subcontractors  in  the  RFP. 


d.  Since  software  costs  are  of  a  high 
risk  nature  and  constitute  a  significant 
cost  factor,  much  can  be  done  during  the 
procurement  process  to  avoid  excessive 
cost.  The  primary  means  for  doing  this 
is  to  limit  the  specifications  of  soft¬ 
ware  requirements  contained  in  the  RFP 
to  that  minimum  consistent  with  the  re¬ 
quired  operational  capabilities  of  the 
ATE  or  TS.  If  excessive  requirements  are 
placed  in  the  RFP  this  will  be  trans¬ 
lated  into  higher  cost  for  the  contrac¬ 
tor.  Further  information  concerning  this 
subject  can  be  found  in  the  Requirements 
Specification  Guidebook,  CDRL  A004. 

e.  One  of  the  primary  requirements  in 
contractor  selection  is  determination  of 
whether  the  contractor  has  sufficient 
understanding  of  USAF  requirements  to 
perform  adequately.  As  previously  indi¬ 
cated,  such  techniques  as  pre-bid  con¬ 
ferences,  plant  visits,  etc.,  are  useful 
means' of  qualifying  contractors.  In  addi¬ 
tion,  however,  it  is  necessary  to  iden¬ 
tify  contractors  who  understand  the 
complexities  of  the  particular  ATE  or  TS 
to  be  procured.  The  procurement  process 
can  facilitate  this  by  requiring  contrac¬ 
tors  to  define  in  their  offerings  high 
technical  risk  software  requirements  and 
describe  how  they  intend  to  approach 
these  requirements.  The  RFP  should 
stress  the  importance  of  responding  with 
methods  and  approaches  which  the 
contractor  proposes  to  employ. 

f.  The  procurement  process  should  be 
conducted  in  such  a  manner  that  the  us¬ 
ing  command  can  assume  operations  and 
maintenance  responsibility  for  the  sys¬ 
tem.  Particular  attention  should  be 
devoted  to  obtaining  necessary  training 
of  USAF  personnel  who  will  operate  and 
maintain  contractor  furnished  ATE  and  TS 
software.  One  scheme  is  to  purchase 
training  and  support  services  using  the 
contracting  method  described  by  ASPR 
3-406.1  (see  paragraph  3. 1.2. 3).  This 
service  would  be  of  a  temporary  nature 
only.  There  is  an  advantage  to  obtaining 
this  service  as  part  of  the  TS  or  ATE 
system  procurement  since  the  contractor 
is  motivated  by  competition  to  avoid 
overpricing  such  services. 
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g.  The  subject  of  contractor  propri¬ 
etary  software  should  be  given  considera¬ 
tion.  It  is  generally  the  case  that 
software  provided  with  ATE  and  TS  is 
largely  a  unique  development  for  the  spe¬ 
cific  system  being  procured  or  the  soft¬ 
ware  is  so  general  it  is  in  the  public 
domain.  Such  software,  since  paid  for  by 
the  government,  becomes  the  property  of 
the  government.  Nevertheless,  contrac¬ 
tors  occasionally  develop  proprietary 
software  which  they  are  reluctant  to 
place  in  the  public  domain.  In  order  to 
avoid  post  contract  award  problems 


caused  by  a  contractor's  reluctance  to 
grant  ownership  of  certain  software  to 
the  government,  because  it  would  violate 
his  proprietary  rights,  specific  pro¬ 
visions  should  be  placed  in  the  RFP.  In 
most  cases,  statements  indicating  that 
all  software  delivered  with  the  system 
become  the  property  of  the  government, 
is  sufficient.  However,  the  form  of  such 
statements,  and  legal  ramifications 
thereof,  are  the  responsibility  of  the 
Procurement  Contracting  Officer  (PCO) 
and  the  acquisition  engineer  should  play 
only  a  technical  advisory  role  in  this 
process. 


Section  4.0  REQUESTS  FOR  PROPOSAL  AND  SUPPORTING  DOCUMENTATION 


This  section  summarizes  the  contracting 
for  software  acquisition  processes  which 
are  concerned  with  preparation  and  re¬ 
lease  of  information  to  qualified  bid¬ 
ders.  The  RFP  contents  are  not  described 
in  detail  herein  since  this  material  is 
included  in  the  Statements  of  Work  and 
Request  for  Proposals  Guidebook,  CDRL 
A006.  Rather,  the  material  included 
herein  is  limited  to  an  overview  of  the 
RFP  with  particular  emphasis  on  its  re¬ 
lationship  with  other  procurement 
activities. 

4.1  REQUIREMENTS  SPECIFICATION 

Requirements  often  referred  to  as  the 
"Technical  Exhibit"  are  the  technical 
heart  of  the  RFP.  These  define  the  de¬ 
tailed  performance  capabilities  which 
the  ATE  or  TS  is  expected  to  perform. 
Trade  studies  and  analyses  necessary  to 
produce  the  Requirements  Specification 
together  with  information  content  is 
contained  in  the  Requirements  Specifi¬ 
cation  Guidebook,  CDRL  A004. 

a.  ASPR  1-1003  requires  a  synopsis 
far  enough  in  advance  of  the  issuance  of 
solicitations,  to  permit  interested 
firms  to  respond.  Several  advantages  are 
provided  to  the  SSEC  by  using  the  vehi¬ 
cle  of  the  preliminary  requirements  spe¬ 
cification  or  technical  exhibit  activity 
as  one  vehicle.  First,  if  draft  spe¬ 
cifications  are  provided  to  interested 
potential  contractors  for  response,  the 
government  benefits  by  having  industry 
involvement  early  in  the  requirements 
specification  phase.  Often  this  involve¬ 
ment  may  provide  information  to  the  SSEC 
which  enables  avoidance  of  excessive 
costs.  ASPR  3-106.1  indicates  this  means 
may  be  used  not  only  to  help  definitize 
requirements,  but  as  a  means  of  reducing 
costs  of  procurement.  If  both  the  SSEC 
and  prospective  contractors  are  thor¬ 
oughly  familiar  with  the  TS  or  ATE  re¬ 
quirements  before  the  RFP  is  released, 
proposal  costs  on  the  part  of  contractor 
is  reduced  and  proposal  evaluation  by 
the  SSEC  is  reduced. 


ASPR  3-106.1  indicates  responses  to  such 
advanced  information  may  be  used  as  an 
aid  to  the  establishment  and  maintenance 
of  qualified  bidders'  lists.  The  quality 
of  response  received  from  a  prospective 
bidder  provides  a  measure  of  his  inter¬ 
est  and  capability.  Care  should  be  exer¬ 
cised,  however,  to  ensure  that  bias  is 
not  inadvertently  admitted  into  the  pro¬ 
curement.  It  is  necessary  to  practice 
impartiality  by  providing  the  same  infor¬ 
mation  to  all  prospective  bidders. 
Requirements  which  tend  to  favor  one  con¬ 
tractor  vs  another,  should  not  be  in¬ 
cluded  in  the  RFP  unless  this  is  done 
deliberately,  not  for  the  benefit  of  the 
contractor  but  for  the  benefit  of  the 
government.  Further,  this  should  only  be 
done  with  full  knowledge  and  concurrence 
of  the  SSA  and  the  PC0. 

b.  In  addition  to  soliciting  evalua¬ 
tion  of  requirements  specification  by 
prospective  bidders,  evaluation  should 
be  solicited  from  the  using  command.  In 
many  cases  using  command  representatives 
are  SSEC  members  so  that  coordination  is 
carried  on  throughout  all  stages  of  the 
procurement. 

4.2  ADDITIONAL  RFP  DOCUMENTATION 

In  addition  to  the  specification  of  re¬ 
quirements,  other  RFP  documentation  with 
which  the  acquisition  engineer  is  con¬ 
cerned  includes  the  Statement  of  Work 
(SOW),  schedule.  Contracts  Data  Require¬ 
ments  List  (CDRL)  and  Data  Item  Descrip¬ 
tions  (DID).  Other  RFP  material  such  as 
instructions  for  preparation  of  cost  pro¬ 
posals,  terms  and  conditions,  etc., 
which  are  prepared  by  the  PC0  and  his 
staff  normally  involves  the  acquisition 
engineer  in  a  consulting  capacity  only. 

4.2.1  Statement  of  Work 

The  statement  of  Work  (SOW)  defines  all 
tasks  to  be  performed  by  the  contractor. 
In  addition  it  describes  each  item  to  be 
delivered  by  the  contractor  and  it  is 
therefore  necessary  that  the  SSEC  ensure 


inclusion  of  all  desired  deliverable 
software  including  the  form  in  which  the 
software  is  to  be  delivered.  Normally 
the  preliminary  SOW  is  made  available 
for  comment  by  prospective  contractors 
along  with  the  requirements 
specification. 

4.2.2  Contract  Data  Requirements  List 

Contract  Data  Requirements  List  (CDRL) 
identifies  the  data,  including  program 
documentation,  status  reports,  technical 
manuals,  etc.,  to  be  provided  by  the  con¬ 
tractor.  It  is  important  that  these  re¬ 
quirements  be  carefully  considered  by 
the  SSEC  prior  to  final  release  of  the 
RFP  since  the  contractor  is  not  obli¬ 
gated  to  supply  any  data  not  included  in 
the  CDRL.  The  Computer  Program  Documenta¬ 
tion  Requirements  Guidebook,  CDRL  A009, 


provides  guideline  factors  useful  in  se¬ 
lecting  a  CDRL  for  TS  and  ATE.  In  addi¬ 
tion  it  provides  checklists  for  ATE  and 
TS  CDRL's.  AFR  310-1,  together  with  its 
AFSC  and  ASD  supplements,  defines  proce¬ 
dures  and  guidelines  applicable  to  data 
acquired  under  contract. 

4.2.3  Data  Item  Descriptions 

Data  Item  Descriptions  (DID)  provide  for¬ 
mat  and  content  of  defined  data  items. 
Complete  DID  lists  are  contained  in  DOD 
5000. 19-L.  The  DID  should  be  tailored  to 
meet  the  requirements  of  individual  pro¬ 
curements  prior  to  use  of  DOD  5000. 19-L 
DID's  for  TS  and  ATE.  Guidance  relative 
to  this  is  included  in  the  Computer  Pro¬ 
gram  Documentation  Requirements  Guide¬ 
book,  CDRL  A009. 
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Section  5.0  SOURCE  SELECTION 


This  section  describes  the  essential 
tasks  performed  by  the  SSEC  to  assist 
the  SSA  in  selection  of  one  or  more  con¬ 
tractors  to  provide  the  ATE  or  TS. 

5.1  ACTIVITIES  PRIOR  TO  RFP  RELEASE 

Figure  5.0-1  illustrates  those  activi¬ 
ties  associated  with  the  source  selec¬ 
tion  process.  Because  most  ATE  and  TS 
systems  are  acquired  using  negotiated 
procurement,  ASPR  references  indicated 
in  the  figure  deal  with  this  procurement 
method. 

5.1.1  Preliminary  Considerations 

Ideally  the  source  selection  process 
originates  with  a  Required  Operational 
Capability  (ROC),  originated  by  the 
using  command  and  submitted  to  the 
Defense  System  Acquisition  Review  Coun¬ 
cil  (DSARC).  For  ATE  or  TS  procured  as 
part  of  a  large  system,  the  system  GOR 
is  provided  and  may  make,  at  most,  mini¬ 
mal  reference  to  the  ATE  or  TS.  However, 
where  the  ATE  or  TS  is  of  sufficient 
magnitude,  or  where  the  required  avail¬ 
ability  date  of  the  ATE  or  TS  is  incon¬ 
sistent  with  the  host  system,  a  GOR  per¬ 
taining  to  the  ATE  or  TS  will  be 
reviewed  by  DSARC.  Functions  and  proce¬ 
dures  associated  with  DSARC  activities 
are  included  in  DODI  5000.2  and  DODD 
5000.26.  The  ROC  results  in  one  or  more 
Program  Management  Directives  (PMD) 
which  provide  specific  guidance  relative 
to  the  ROC  and  directs  any  additional 
study  or  developmental  work  deemed 
necessary  by  DSARC.  Where  the  ROC  and 
associated  study /developmental  activity, 
together  with  recommendations  by  DSARC 
results  in  OSD  and/or  Headquarters  USAF 
DCS/R&D  approval,  a  Decision  Coordi¬ 
nating  Paper  (DCP)  is  issued  which  summa¬ 
rizes  the  ATE/TS  program  features  and 
objectives  and  defines  authority  dele¬ 
gated  within  the  USAF  for  management  of 
the  program. 

Following  these  actions  an  SSA  is  desig¬ 
nated,  an  SSEC  (or  SSAC  and  SSEB)  formed 


and  the  plan  described  in  paragraph  3.2 
is  prepared. 

5.1.2  ASPR  1003  Requirements 

ASPR  1003  requires  public  notice  of  pend¬ 
ing  government  procurements.  It  pre¬ 
scribes  that  synopsis  of  the  proposed 
procurement  be  published  in  Commerce 
Business  Daily  not  later  than  10  days 
preceding  release  of  the  RFP.  This  al¬ 
lows  potential  bidders  who  are  not  on 
the  current  bidder's  list  to  request  the 
RFP.  The  SSEC  prepares  the  necessary  syn¬ 
opsis  which  is  then  released  through  the 
PCO. 

5.1.3  ASPR  1.903  and  3-802.2  Require¬ 
ments 

ASPR  1.903  and  3-802.2  prescribe  require¬ 
ments  for  the  creation  and  maintenance 
of  qualified  bidder's  lists.  While  it  is 
in  the  best  interest  of  the  USAF  to 
encourage  competition,  it  is  neverthe¬ 
less  wasteful  of  both  government  and 
industry  resources  to  encourage  prospec¬ 
tive  contractors  to  prepare  proposals  if 
they  are  not  qualified  to  perform  satis¬ 
factorily  on  the  contract.  AFR  70-15 
specifies  that,  in  developing  source 
lists,  all  available  sources  should  be 
considered.  Of  those  sources  readily 
available  to  the  ATE  or  TS  SSEC,  pre-re¬ 
lease  conferences  with  prospective  bid¬ 
ders,  including  optional  plant  visits  by 
SSEC  members,  and  evaluation  of  poten¬ 
tial  bidder's  responses  to  preliminary 
RFP  material,  are  probably  the  most 
useful. 

Of  particular  concern  is  the  software  ca¬ 
pability  of  the  prospective  bidder.  Some 
points  to  keep  in  mind  are: 

a.  Does  this  potential  bidder  have  an 
adequate  staff  of  skilled  programmers 
knowledgeable  on  ATE  or  TS  from  which  to 
draw  a  nucleus?  If  the  potential  bidder 
intends  to  recruit  and  train  the  neces¬ 
sary  software  staff  in  the  event  he  wins 
the  contract,  it  is  doubtful  he  is  a 
qualified  bidder  in  the  sense  of  ASPR 
requirements. 
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b.  Does  the  contractor  have  adequate 
software  development  and  support  facili¬ 
ties?  It  is  often  the  case  with  ATE  or 
TS  that  the  computers  to  be  delivered 
with  the  system  can  be  used  for  software 
development.  However,  particularly  in 
the  case  of  ATE,  the  system  computer  may 
not  be  equipped  with  necessary  periph¬ 
eral  devices  to  support  such  tasks  as 
ATLAS  language  compilation  in  an  effi¬ 
cient  manner.  Further,  it  is  necessary 
to  have  adequate  key  punch  and  verifica¬ 
tion  facilities,  peripheral  support 
printers,  etc.  to  aid  program  checkout. 
If  the  contractor  is  intending  to  pur¬ 
chase/lease  all  required  support  equip¬ 
ment  and  dedicate  it  to  the  ATE  or  TS 
job  at  hand  it  is  doubtful  he  can  be  com¬ 
petitive.  Normally  computer  support  fa¬ 
cilities  are  shared  between  a  number  of 
contractor  programs  and,  therefore,  the 
ATE  or  TS  contract  should  be  burdened 
with  only  its  share  of  the  cost  of  such 
facilities. 

c.  Is  the  management  structure  of  the 
company  conducive  to  the  contract?  The 
company,  if  a  serious  contender,  will 
have  established  a  program  or  proposal 
manager,  a  software  manager  and  at  least 
a  small  staff  of  engineers  well  in  ad¬ 
vance  of  the  date  the  RFP  is  to  be 
released.  If  by  the  time  the  bidder's 
list  is  being  prepared,  the  contractor 
has  not  done  these  things,  the  company 
may  not  be  sufficiently  committed  to  the 
job  to  warrant  serious  consideration. 

d.  Does  the  contractor  have  adequate 
prior  ATE/TS  system  experience?  AFR 
70-15  indicates  the  screening  criteria 
should  include  past  and  present  experi¬ 
ence  regarding  performance  capability  in 
such  areas  as  logistics  support,  finan¬ 
cial  status  and  (software)  production 
capacity. 

e.  Final  approval  of  the  screening 
criteria  to  be  applied  to  any  source 
list  is  approved  by  the  SSA.  It  is  actu¬ 
ally  the  PCO  who  determines  sources  and 
maintains  the  source  list.  Thus  in  estab¬ 
lishing  lists  the  ATE  or  TS  acquisition 
engineering  role  is  advisory  in  areas 
such  as  those  indicated  above. 


5.1.4  Final  Considerations 

Advanced  ATE  and  TS  procurement  informa¬ 
tion  is  prepared  and  released  in  accor¬ 
dance  with  ASPR  3-106.2.  This  includes 
advanced  RFP  material,  particularly  the 
Statement  of  Work  (SOW)  and  the  techni¬ 
cal  specifications  as  indicated  in  para¬ 
graph  4.1.  This  is  a  useful  process  both 
to  the  government  and  prospective  bid¬ 
ders.  Further,  it  is  an  excellent  vehi¬ 
cle  for  establishing  useful  dialog 
between  the  SSEC  and  prospective  bid¬ 
ders.  The  ASPR's  are  specific  in  their 
requirement  to  ensure  that  any  item  of 
prerelease  information  provided  to  one 
potential  bidder  is  equally  available  to 
all.  This  can  easily  be  satisfied  by, 
for  example,  making  the  same  version  of 
the  advanced  RFP  available  to  all  pro¬ 
spective  contractors. 

ASPR  3-106.2  specifies  that,  when  this 
pre-solicitation  approach  is  used,  the 
advanced  information  is  released  through 
the  PCO  to  all  known  potential  contrac¬ 
tors.  Further,  the  contractors  are 
requested  not  only  to  respond  with  com¬ 
ments  and  suggestions,  but  are  requested 
to  provide  an  expression  of  interest  in 
the  contemplated  procurement.  Also,  ASPR 
3-106.2  indicates  the  prospective  con¬ 
tractor  may  be  requested  to  provide 
information,  such  as  management,  engi¬ 
neering,  software,  etc.  capability. 

5.2  PROPOSAL  DEVELOPMENT  AND 
EVALUATION 

This  activity  is  initiated  by  the  PCO 
when  he  formally  releases  the  RFP. 

5.2.1  Pre-Proposal  Conference 

In  most  cases  a  pre-proposal  conference 
will  be  held  at  the  procuring  agencies' 
facilities  shortly  after  release  of  the 
RFP.  ASPRS  3-504  and  3-106.2  provide  re¬ 
quirements  for  conduct  of  this  meeting. 
The  meeting  is  chaired  by  the  PCO  with 
the  SSEC  chairman  in  attendance.  Other 
members  of  the  SSEC  will  be  present  as 
needed.  This  meeting,  attended  by  all 
prospective  bidders,  provides  the  oppor- 
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t unity  for  bidders  to  ask  questions  of 
clarification.  Since  each  bidder  is  re¬ 
presented,  all  are  witness  to  aqy  ques¬ 
tions  and  the  answers  provided  by  the 
government.  In  many  cases  bidders  are 
instructed  to  direct  any  further  ques¬ 
tions  subsequent  to  the  meeting  in  writ¬ 
ing  through  the  PCO.  In  this  way 
questions  can  be  prepared  and  submitted 
by  the  PCO  to  all  bidders. 

5.2.2  Formal  RFP  Release 

It  is  normal  practice  that  informal  dia¬ 
log  between  bidders  and  SSEC  members  be 
terminated  with  formal  RFP  release.  This 
is  done  to  ensure  that  no  bidder  is 
availed  of  information  which  is  not 
available  to  every  other  bidder.  Conse¬ 
quently  any  information  exchange  between 
bidders  and  the  SSA/SSEC  should  be  con¬ 
strained  to  be  handled  formally  (in  writ¬ 
ing)  and  information  exchanges  should  be 
made  through  the  PCO. 

5.2.3  ICE  Preparation 

AFR  709-15  prescribes  the  preparation  of 
the  Independent  Cost  Estimate  (ICE).  It 
discourages  release  of  this  information 
to  prospective  bidders.  To  do  so  might 
influence  the  bidder  to  make  his  finan¬ 
cial  offering  against  the  ICE  rather 
than  against  his  proposed  system. 

The  ATE  or  TS  software  acquisition  engin¬ 
eer  plays  a  key  role  in  producing  the 
ICE  since  it  is  his  estimate  which  is 
used  for  the  software  ICE.  The  Cost  Mea¬ 
suring  and  Reporting  Guidebook,  CDRL 
A003,  contains  several  cost  estimating 
schemes  which  may  be  used  for  this  pur¬ 
pose.  It  is  fundamental,  however,  to 
each  of  these  schemes  that  an  element  of 
judgement  is  involved.  For  example,  sev¬ 
eral  of  these  methods  require  estimation 
of  the  number  of  computer  instructions 
to  be  developed  for  the  system  to  be  pro¬ 
cured.  The  validity  of  the  ICE  produced 
is  constrained  to  the  degree  of  validity 
of  this  estimate.  Consequently  this 
judgement  should  be  exercised  carefully 
and  be  based  on  as  much  actual  experi¬ 
ence  with  similar  systems  previously 
procured  as  is  reasonably  available. 


5.2.4  Proposal  Evaluation 

AFR  70-15  and  its  supplements  provide  di¬ 
rection  concerning  the  analysis  and  eval¬ 
uation  of  bidder's  proposals.  This 
process  consists  of  scoring  and  evalua¬ 
ting  each  proposal,  determining  those 
contractors  who  are  in  the  competitive 
range,  and  the  identification  and  handl¬ 
ing  of  proposal  deficiencies. 

5. 2.4.1  Specific  Evaluation  Criteria. 
As  previously  indicated,  specific  evalua¬ 
tion  criteria  will  have  been  placed  in 
the  source  selection  plan.  The  process 
of  proposal  evaluation  is  the  process  of 
determining  whether  the  proposal  meets 
the  requirements  of  the  RFP  in  light  of 
the  specific  evaluation  criteria.  The 
following  guidelines  should  be  consid¬ 
ered  in  this  process. 

a.  The  acquisition  engineer  need  not 
accept,  without  question,  data  presented 
in  the  prooosal.  He  should  use  his  know¬ 
ledge  and  experience  to  determine  feasi¬ 
bility,  logic  and  reasonableness  of  the 
contractor's  response.  If  in  doubt  con¬ 
sultants  should  be  used  to  assist  the 
evaluator. 

b.  The  evaluator  should  have  a  de¬ 
tailed  understanding  of  the  SOW  and  tech¬ 
nical  annex.  These  things  provide 
minimum  acceptable  response.  Where  a  bid¬ 
der  is  found  deficient  this  deficiency 
should  be  recorded  along  with  the  rea¬ 
sons  why  this  deficiency  judgement  was 
made.  Such  information  is  extremely  im¬ 
portant  in  the  event  of  contractor 
protest  actions. 

c.  The  applicable  USAF  command  media 
documentation  allows  latitude  on  the 
part  of  evaluators  to  perform  the  evalua¬ 
tion  function.  It  is  recognized  each 
evaluator  will  approach  the  task  based 
on  his  own  judgement  and  experience.  It 
is,  however,  important  to  evaluate  in  a 
consistent  manner.  That  is,  the  methods 
and  biases  assumed  by  an  evaluator 
should  be  applied  uniformly  to  all 
proposals. 


d.  The  evaluation  process  results  in 
a  narrative  report.  This  report,  submit¬ 
ted  to  the  SSEC  chairman  and  available 
for  SSA  review  as  required,  should  be 
specific  rather  than  general  in  nature. 
The  report  should  indicate  what  was  of¬ 
fered;  how  it  met  the  standard,  or  how 
it  failed  to  meet  the  minimum  require¬ 
ments;  what  must  be  done  to  remedy  the 
deficiency;  and  what  impact  (including 
technical  and  cost  risk)  the  deficiency 
correction  will  have  on  the  overall  ac¬ 
ceptability  of  the  contractor's  pro¬ 
posal. 

e.  Identification  and  assessment  of 
risk  are  important  parts  of  the  evalua¬ 
tion  process.  Certain  risks  may  be  inher¬ 
ent  in  the  TS  or  ATE  software  because  of 
the  particular  system  requirements,  or 
risks  may  occur  as  a  result  of  a  particu¬ 
lar  contractor's  approach  to  satisfac¬ 
tion  of  those  requirements.  In  making 
the  risk  determination,  the  evaluator 
should  judge  the  probability  of  success 
and,  in  the  event  of  failure,  what  alter¬ 
natives  are  available  to  overcome  the 
risks. 

5. 2. 4. 2  Proposal  Scoring.  AFR  70-15 
suggests  schemes  for  scoring  proposals. 
Two  types  of  scoring  are  described;  "nu¬ 
merical"  (objective)  and  "narrative" 
(subjective). 

a.  The  numerical  method  involves  the 
assignment  of  a  pre-established  numeri¬ 
cal  scale  to  the  designated  evaluation 
factors. 

b.  The  narrative  approach  does  not  re¬ 
quire  the  assignment  of  numerical  scores 
below  the  item  level.  It  provides  the 
evaluation  team  personnel  with  suffi¬ 
cient  flexibility  to  use  their  judgement 
as  to  whether  the  collective  factors 
met,  failed  to  meet,  or  exceed  the  stan¬ 
dard,  and  thus  determine  the  score  for 
the  item  as  a  whole.  In  this  way,  seri¬ 
ous  deficiencies  impacting  the  item  can 
be  given  proper  consideration  in  deter¬ 
mining  the  score  assigned. 


The  scoring  of  ATE  and  TS  software  pro¬ 
posals  frequently  involves  a  combination 
of  these  methods.  The  degree  to  which  a 
contractor's  approach  to  a  particular 
software  requirement  may  be  scored 
numerically.  However,  a  judgement  of  the 
adequacy  of  the  proposed  software  organi¬ 
zation,  or  the  proposed  allocation  of 
software  tasks  to  the  operating  system 
vs  "application"  programs,  or  the  merits 
of  a  data  management  scheme  may  best  be 
evaluated  subjectively. 

5. 2.4.3  Proposal  Coding.  AFR  70-15 
suggests  use  of  symbols  or  color  codes 
as  an  aid  to  determine  how  well  the  com¬ 
panies'  proposal  met  the  standard  for 
each  factor. 

A  "+"  sign  can  be  used  to  indicate  that 
the  contractor  has  exceeded  minimum  re¬ 
quirements  and  a  "  "  to  indicate  that 
the  contractor  has  met  minimum  require¬ 
ments;  and  a  to  indicate  that  he  has 
failed  to  meet  minimum  requirements  for 
each  factor  evaluated. 

In  using  a  color  code  scheme,  the  follow¬ 
ing  spectrum  should  be  used: 

a.  Exceptional,  or  exceeds  standards- 

-green; 

b.  Meets  standards — blue; 

c.  Marginal,  or  fails  to  meet 

standards--yel low; 

d.  Unacceptable — red. 

Upon  completion  of  the  factor  evaluation 
of  the  item,  evaluators  can  determine 
the  designator,  from  the  scale  presented 
below,  which  best  reflects  the  extent  to 
which  the  contractor  has  met  the  overall 
item  requirements.  The  scale  can  be 
devised  as  follows: 

Evaluation  Scale  Color 

10— Exceptional  Green 

9 

—Exceeds  Standards 

8 
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7 

6 — Meets  Standards  Blue 

5 

4 

3 

--Fails  to  Meet  Standards  Yellow 

2 

1 

0 — Unacceptable  Red 

Scores  may  be  applied  as  follows: 

a.  If  the  majority  of  the  factors 
meet  standards,  the  requirement  is  not 
overly  difficult  to  meet,  and  the  fac¬ 
tors  which  are  deficient  are  of  a  very 
minor  nature  or  are  susceptible  to  easy 
correction,  the  item  should  be  scored 
"5". 

b.  If  the  requirement  is  relatively 
difficult  to  meet  and  the  majority  of 
the  factors  are  acceptable,  no  major 
deficiencies  or  risks  exist  therein,  and 
the  collective  approach  yields  a  qualita¬ 
tive  benefit  to  the  government  beyond 
that  which  is  minimal,  a  score  of  6  or  7 
should  be  assigned,  dependent  upon  the 
benefits  to  be  attained. 

c.  If  the  major  number  of  important 
factors  are  acceptable  but  one  or  more 
of  the  factors  are  deficient  and  some 
minor  risk  is  involved  in  the  correction 
thereof,  the  score  for  the  time  should 
be  "4". 

d.  If  a  majority  of  the  factors  for 
the  item  are  deficient  and  their  correc¬ 
tion,  either  collectively  or  individu¬ 
ally,  poses  a  serious  problem  in 
correction,  or  has  a  "domino"  effect  on 
other  design  features,  or  the  approach 
poses  a  high  risk  without  means  for 
correction,  if  the  approach  fails,  a 
score  of  "3,  2,  1"  should  be  assigned 


with  the  lower  score  indicating  a  seri¬ 
ous  or  severe  condition. 

e.  If  the  major  factors  of  the  item 
are  deficient  to  the  extent  that  a  major 
reorientation  of  the  proposal  is  neces¬ 
sary,  or  if  the  approach  taken  is  unde¬ 
sirable  and  correction  would  require  a 
major  reproposal,  the  item  should  be 
scored  "0". 

f.  If  the  requirement  is  particularly 
difficult  to  meet  and  the  proposal  of¬ 
fers  an  approach  which,  with  little  or 
no  risk,  will  yield  a  result  which  ex¬ 
ceeds  requirements  qualitatively,  the 
item  should  be  scored  "8,  9  and  10"  de¬ 
pendent  upon  the  level  of  the  exception¬ 
al  features  offered. 

g.  In  assigning  numerical  scores,  it 
is  imperative  that  a  scoring  above  5  not 
be  assigned  simply  because  a  proposal  of¬ 
fers  to  exceed  a  requirement  in  some 
quantitative  way.  A  simulation  fidelity 
or  ATLAS  processing  capability  beyond 
that  required  for  a  given  TS  or  ATE  is 
not  necessarily  an  advantage  if  this  ex¬ 
cess  capability  will  not  substantially 
enhance  the  TS  or  ATE  system.  The  assign¬ 
ment  of  score  above  "5"  should  reflect 
some  qualitative  achievement,  such  as 
improved  system  operability. 

h.  In  some  instances,  solicitations 
will  indicate  a  minimum  requirement  for 
certain  aspects  of  the  system,  but  will 
express  a  "goal"  beyond  that  point  which 
is  minimally  required.  In  those  in¬ 
stances,  scores  above  "5"  should  be 
assigned  when  a  proposal  states  and  the 
analysis  verifies  that  something  beyond 
the  required  quantity  can  be  attained. 

i.  AFR  70-15  indicates  the  greatest 
problem  posed  by  the  application  of  any 
evaluation  approach,  us’ng  numbers  denot¬ 
ing  how  well  a  proposal  meets  the  factor 
standard,  is  the  conservativeness  dis¬ 
played  by  evaluators. 

(1)  While  there  is  no  need  to  ex¬ 
aggerate  findings,  it  is  imperative  that 
the  evaluator  take  advantage  of  the  full 
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range  of  scores  if  circumstances  war¬ 
rant,  so  that  the  variances  between  pro¬ 
posals  may  be  readily  identified.  The 
evaluation  process  does  not  attempt  to 
classify  all  proposals  as  either  fully 
acceptable  or  as  fully  unacceptable.  One 
reason  for  the  scrutiny  to  which  the  pro¬ 
posals  are  subjected  is  to  develop  the 
proper  numerical  scoring  for  the  item  be- 
ween  the  maximum  (10)  and  the 
unacceptable  (0). 

(2)  If  the  evaluator  takes  the 
easy  way  out  by  considering  all  propos¬ 
als,  regardless  of  their  ingenuity  or 
lack  thereof,  as  "meeting  minimum  re¬ 
quirements,"  a  major  purpose  of  the  eval¬ 
uation  effort  is  lost.  When  a  contractor 
meets  minimum  requirements  in  a  way 
which  is  simple  and  effective,  he  should 
be  given  more  credit  for  that  effort 
than  if  he  meets  requirements  for  a  task 
in  a  complex  and  difficult  manner.  In 
other  words,  evaluators  should  strive 
for  a  realistic  score  along  the  numeri¬ 
cal  scale  which  is  truly  representative 
of  the  contractor's  ability  to  satisfy 
the  requirement. 

5.2.5  Deficiencies  Evaluation 

While  the  analysis  of  the  contractors' 
proposals  pertaining  to  a  factor  of  the 
evaluation  is  being  made,  the  evaluator 
must  perform  another  task.  The  evaluator 
must  record,  separately  and  in  addition 
to  the  narrative  analysis,  the  deficien¬ 
cies  found  in  each  contractor's  pro¬ 
posal.  Since  the  individual  evaluators 
cannot  determine  during  the  course  of 
the  evaluation  which  of  the  contractors 
will  be  selected  in  the  final  competi¬ 
tion,  it  is  important  that  he  record  in 
an  unbiased  manner  the  deficiencies 
found  and  cite  the  effect  the  uncor¬ 
rected  proposal  will  have  on  the  pro¬ 
gram.  This  in  turn  provides  the 
contractors  with  the  opportunity  to  pro¬ 
pose  corrective  action,  and  if  appropri¬ 
ate,  correct  their  statement  of  work, 
end  item  specifications,  schedules,  and 
costs. 


5.2.5. 1  Minimum  Requirements.  For  the 
purposes  of  source  selection  actions  AFR 
70-15  defines  "deficiency"  as  ary  part 
of  a  contractor's  proposal  which  fails 
to  meet  the  government's  minimus1  require¬ 
ments  established  in  the  solicitation. 
Any  proposal  part,  which,  when  compared 
to  the  standard  described  for  a  factor; 
(1)  fails  to  meet  the  minimum  require¬ 
ments  represented  by  the  standard,  (2) 
proposes  an  approach  which  poses  unac¬ 
ceptable  risk,  or  (3)  is  an  omission  of 
data  which  makes  it  impossible  to  assess 
compliance  with  the  standard  for  that 
requirement,  is  considered  deficient. 

5. 2. 5. 2  Design  Approach.  In  some  in¬ 
stances  a  deficiency  may  describe  an 
approach  taken  by  a  contractor  in  the  de¬ 
sign  of  its  system  which  yields  perfor¬ 
mance  which  is  not  desired.  Since  the 
selection  of  a  source  is  dependent  upon 
the  combined  findings  of  technical,  man¬ 
agement,  production,  logistics,  opera¬ 
tional  aspects,  and  cost  considerations, 
it  is  possible  that  a  proposal  which  ini¬ 
tially  contained  several  deficiencies 
may  eventually  be  selected  as  the  succes¬ 
sful  one  by  the  SSA.  For  this  reason,  de¬ 
ficiencies  should  be  accurately  reported 
by  the  evaluators  so  that  they  may  be 
resolved  by  the  bidding  contractor. 

5. 2. 5. 3  Comparative  Evaluation.  Defi¬ 
ciencies  identified  derive  only  from 
individual  evaluation  of  each  contrac¬ 
tor's  proposal  against  the  evaluation 
criteria/standard,  i.e.,  the  govern¬ 
ment's  minimum  requirements  for  the 
solicitation.  In  no  event  should  defi¬ 
ciencies  derive  from  a  comparative  eval¬ 
uation  of  the  relative  strengths  and 
weaknesses  of  competing  contractor's 
proposals. 

5.2.6  Determination  of  Competitive 
Range 

By  law  (10  U.S.C.  2301(g))  written  or 
oral  discussions  must  be  conducted  with 
all  responsible  contractors  who  submit 
proposals  within  a  competitive  range. 
The  PC0  makes  this  determination  with 
support  of  the  SSEC. 
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5. 2.6.1  Competitive  Range  Evaluation. 
The  competitive  range  must  be  deter¬ 
mined,  after  evaluation  of  all  proposals 
received,  on  the  basis  of  price  or  cost, 
technical,  and  other  salient  factors,  in¬ 
cluding  proposal  deficiencies  and  their 
potential  for  correction,  and  must  in¬ 
clude  all  proposals  which  have  a  reason¬ 
able  chance  of  being  selected.  The 
objective  is  not  to  eliminate  proposals 
from  the  competitive  range;  on  the  con- 
tray,  it  is  to  conduct  written  or  oral 
discussions  to  enhance  proposal  accepta¬ 
bility,  thereby  facilitating  competition 
and  to  obtain  proposals  from  which  award 
may  be  made  in  the  government's  best  in¬ 
terests,  price  or  cost,  technical,  and 
other  salient  factors  considered. 

5. 2. 6. 2  Competitive  Range  Exceptions. 
After  evaluation  of  all  proposals,  a  pro¬ 
posals)  may  be  considered  outside  the 
competitive  range  either  initially,  with¬ 
out  written  or  oral  discussions,  or 
after  written  or  oral  discussions  with  a 
contractor^ )  whose  initially  evaluated 
proposal (s)  appeared  to  have  a  reason¬ 
able  chance  of  being  selected  for  award. 
The  following  are  examples  of  conditions 
on  which  a  determination  may  be  made 
that  a  proposal  does  not  have  a 
reasonable  chance  of  being  selected  for 
award. 

a.  It  does  not  reasonably  address  the 
essential  requirements  of  the  solicita¬ 
tion. 

b.  In  research  or  development  procure¬ 
ment,  when  a  substantial  design  drawback 
is  apparent  in  the  proposal,  and  suffi¬ 
cient  correction  or  improvement  to  con¬ 
sider  the  proposal  further  would  require 
virtually  an  entirely  new  technical 
proposal . 

c.  The  proposal  contains  major  techni¬ 
cal  or  business  deficiencies  or  omis¬ 
sions,  or  out-of-line  costs,  which 
initial  or  continuing  discussions  with 
the  contractor  could  not  resonably  be 
expected  to  cure.  Contractors  whose  pro¬ 
posals  are  determined  to  be  outside  the 
competitive  range  and  with  whom  initial 


or  continuing  discussions  are  not  to  be 
conducted,  must  be  so  notified  promptly 
in  accordance  with  ASPR  3-508.2. 

5. 2. 6. 3  Competitive  Range  Notifica¬ 
tion.  AFR  70-15  indicates  all  contrac¬ 
tors  determined  to  be  in  the  competitive 
range  and  selected  to  participate  in 
oral  or  written  discussions  must  be  ad¬ 
vised  of  any  deficiencies  in  their  pro¬ 
posals,  and  offered  a  reasonable 
opportunity  to  correct  or  resolve  the 
deficiencies.  It  further  indicates; 

a.  Deficiencies  must  derive  from  an 
individual  evaluation  of  each  contrac¬ 
tor's  proposal  against  the  evaluation 
criteria/RFP.  Care  should  be  exercised 
to  ensure  deficiencies  are  not  derived 
from  a  comparison  of  one  contractor's 
proposal  with  another. 

b.  Discussions  should  be  conducted  in 
a  way  so  as  to  avoid  inadvertent  dis¬ 
closure  of  competing  contractor's 
strength's,  weaknesses,  technical  infor¬ 
mation  or  ideas. 

5.2.7  Contract  Def initization  and 
Price  Negotiation 

Depending  upon  whether  conventional  or 
4-step  source  selection  is  employed 
prices  are  negotiated  and  contracts  are 
definitized  with  one  or  more  potential 
contractors. 

The  acquisition  engineer's  role  in  this 
process  is  primarily  required  during  the 
Fact  Finding  process.  During  this  pro¬ 
cess  the  SSEC  must  participate  in  these 
negotiations  as  necessary  to  support  the 
PCO  in  his  efforts  to  negotiate  the  con¬ 
tract  which  is  most  advantageous  to  the 
government.  In  so  doing  the  SSEC  will, 
if  necessary,  challenge  the  proposed  con¬ 
tract  which  is  most  advantageous  to  the 
government.  In  so  doing  the  SSEC  will, 
if  necessary,  challenge  the  proposed  con¬ 
tractor's  software  planning  including 
his  resource  estimates  for  each  software 
subsystem.  In  negotiated  procurements 
the  acquisition  engineer  should  be  as 
much  concerned  with  contractor  estimates 


which  are  considered  too  low  as  well  as 
those  considered  too  high.  Unreasonable 
cost  estimates,  either  high  or  low,  are 
usually  indicative  of  lack  of  understand¬ 
ing  of  the  job  by  the  contractor. 

5.3  SELECTION  OF  CONTRACTORS 

Following  contract  definitization  the 
SSEC  prepares  an  evaluation  report  and 
accompanying  presentation  to  the  SSA. 
This  report  and  presentation  conveys  the 
results  and  significant  points  of  the 
evaluations  and  makes  recommendations 
concerning  awards.  Following  his  review 
the  SSA  makes  a  final  decision. 

This  decision  is  documented  in  the 
source  selection  Decision  Document  in 
accordance  with  ASPR  S2-102. 1 ( xi i i ).  No¬ 
tification  of  awards  is  made,  both  to 
successful  and  unsuccessful  bidders,  by 
the  PCO.  ASPR  3-508  prescribes  the  re¬ 
quired  notifications.  AFSC  Sup  1,  AFR 
70-15,  contains  guidance  on  debriefing 
of  contractor  in  accordance  with  ASPR 
3-508. 

Debriefing  of  unsuccessful  bidding  con¬ 
tractors  is  required  whenever  a  contract 
award  is  made  on  some  basis  other  than 
price  and  its  purpose  is  to  assist  con¬ 
tractors  in  upgrading  their  future  pro¬ 
posals.  Here  the  basic  intent  is  to  tell 
the  unsuccessful  contractor  where  his 
proposal  was  considered  weak  or  non-re- 
sponsive  in  relation  to  the  evaluation 
criteria.  An  excellant  source  of  such 
information  is  the  deficiency  reports 
which  were  not  fully  addressed  by  the 
contractor  and  resolved  to  the  govern¬ 
ment's  satisfaction. 

A  final  source  selection  activity  of  the 
SSEC  is  preparation  of  a  Lessons  Learned 
documentation. 

5.3.1  Documentation  Preparation 

AFR  70-15  and  its  supplements  define  re¬ 
quirements  on  SSEB  and  SSAC  evaluation 
and  analysis  reports  and  associated  pre¬ 
sentations.  Since  for  the  majority  of 
ATE  and  TS  procurements,  SSEB  and  SSAC 


functions  are  combined  under  the  SSEC, 
one  analysis  report  is  normally  pro¬ 
duced.  However,  the  format  and  contents 
provided  in  Attachment  4,  AFR  70-15,  are 
applicable. 

a.  The  information  provided  contains 
a  summation  of  the  evaluation  and  narra¬ 
tion  of  both  scored  (quantitative)  and 
unscored  (qualitative)  evaluation  items. 
AFR  70-15  suggests  making  each  contrac¬ 
tor  assessment  precise.  It  should  high¬ 
light  strengths,  weaknesses  and  risks 
associated  with  each  proposal.  It  also 
suggests  reporting  cost  information  in 
two  separate  sections. 

b.  Items  should  be  presented  in  un¬ 
weighted,  or  raw  score  form.  The  end  re¬ 
sult  is  an  evaluation  of  each  proposal 
against  solicitation  requirements  based 
on  evaluation  criteria/standards.  No  com¬ 
parison  of  one  contractor  against  the 
standard  of  another  contractor's  offer¬ 
ing  should  be  made. 

c.  The  report  should  portray,  in  nar¬ 
rative  form,  the  evaluation  results  of 
each  proposal  as  originally  submitted, 
as  well  as  results  of  deficiency  disposi¬ 
tion  and  of  negotiation  actions. 

d.  Attachment  3,  AFR  70-15  provides  a 
format  for  reporting  the  disposition  of 
deficiencies.  Generally  this  includes, 
along  with  an  identification  of  the  con¬ 
tractor  and  the  evaluation  item,  a  de¬ 
scription  of  the  nature  of  the 
deficiency,  the  effect  of  the  deficiency 
on  the  program  if  accepted  "as  is"  and 
references  which  substantiate  that  the 
offering  is  deficient.  This  is  approved 
by  the  evaluator  and  SSEC  chairman. 

5.3.2  Lessons  Learned  Report 

AFR  70-15  requires  submittal  of  a  Les¬ 
sons  Learned  Report  within  8  weeks  after 
the  SSA  decision.  AFSC  Sup  1  requires 
submittal  within  45  days  after  contract 
award.  This  report  is  prepared  by  the 
SSEC  for  submittal  to  the  SSA.  It  covers 
those  particular  events  which  occurred 


during  the  source  selection  activity 
which  would  be  helpful  in  avoiding  pit- 
falls  in  future  procurements.  The  refer¬ 
enced  regulations  are  non-specific  on  a 
particular  format  or  the  detailed  con¬ 
tent  requirements  of  this  report. 


Section  6.0  ACQUISITION  MANAGEMENT 


Contracting  for  software  acquisition  con¬ 
tinues  beyond  the  source  selection  pro¬ 
cess.  This  section  summarizes  this 
continuing  activity. 

6.1  POST  AWARD  ACTIVITIES 

The  primary  task  of  the  acquisition  engi¬ 
neer  subsequent  to  contract  award  is  sup¬ 
port  of  the  SPO  Director  in  his 
management  of  the  acquisition  from  the 
standpoints  of  cost,  schedule  and  techni¬ 
cal  integrity.  These  subjects  are 
covered  in  the  Cost  Measuring  and  Report¬ 
ing  Guidebook,  CDRL  AQ03,  the  Measuring 
and  Reporting  Software  Status  Guidebook, 
CDRL  A008,  the  Reviews  and  Audits  Guide¬ 
book,  CDRL  AOOF  and  the  Management  Re¬ 
porting  Guidebook,  CDRL  AOOG.  This 
paragraph  provides  an  overview  of 
applicable  regulations. 

6.1.1  AFR  800-6  Requirements 

AFR  800-6  provides  policy,  establishes 
reporting  requirements,  assigns  responsi¬ 
bility  and  establishes  procedures  for 
the  application  of  financial  management 
control  techniques  to  Air  Force  acquisi¬ 
tion  and  modifications  programs.  While 
it  is  applicable  only  to  major  programs, 
its  concepts  are  nevertheless  applicable 
to  smaller  programs.  In  addition,  TS  and 
ATE  systems  procured  as  integral  parts 
of  larger  systems  are  subject  to  the 
letter  of  AFR  800-6. 

The  technique  of  AFR  800-6  involves 
structuring  a  larger  program  into 
smaller  more  manageable  elemental  tasks 
by  virtue  of  a  Work  Breakdown  Structure 
(WBS).  Under  this  process  the  software 
will  have  been  categorized  into  its  prin¬ 
cipal  components.  AFR  800-17  provides 
general  guidance  for  WBS  development  and 
MIL-STD-881A  provides  a  standard  method 
for  structuring  a  program  summary  WBS. 

The  WBS  provides  the  initial  key  to  suc¬ 
cessful  management  by  the  SPO.  As  previ¬ 
ously  indicated  it  reduces  the  complex 
program  to  a  number  of  smaller,  more  man¬ 


ageable  tasks.  The  entire  system  by 
which  contractor  cost,  schedule  and  per¬ 
formance  data  is  obtained  and  evaluated 
by  the  USAF  is  based  on  the  WBS. 

AFR  800-6  thereby  further  establishes 
the  methods  and  requirements  for  plan¬ 
ning  for  cost  expenditures  by  the  con¬ 
tractor,  reporting  of  expenditures 
against  these  plans  and  management  of 
contractor  financial  activities. 

Of  particular  concern  to  the  acquisition 
engineer  is  the  analysis  of  contractor 
supplied  software  cost  data  where  the 
ATE  or  TS  has  been  procured  on  a  cost 
reimbursement  basis. 

Further,  even  for  fixed  price  contracts, 
acquisition  engineer  awareness  of  the 
actual  software  costs,  through  use  and 
understanding  of  the  financial  informa¬ 
tion  provided  by  the  contractor,  is 
important.  Frequently  Supplemental  Agree¬ 
ments  (SA)  will  be  negotiated  when 
changes  occur.  TS  and  ATE  systems  for 
new,  evolving  weapon  systems  are  suscep¬ 
tible  to  significant  changes  in  require¬ 
ments.  The  knowledge  of  actual  cost  is 

an  extremely  useful  aid  to  determination 
of  reasonableness  of  software  estimates, 
and  negotiating  price  increases  with  the 
contractor. 

6.1.2  AFR  800-14  Requirements 

AFR  800-14  establishes  policy  for  the 

acquisition  and  support  of  computer 
equipment  and  software  included  as  inte¬ 
gral  parts  of  other  systems.  It  estab¬ 
lishes  for  formal  reviews,  both  to 
determine  schedule  and  technical  perform¬ 
ance.  It  requires  that  engineering  and 
technical  direction  of  engineering  ef¬ 
forts  must  be  reviewed  on  a  periodic 

basis  to  determine  the  technical  ade¬ 
quacy  of  contractor  efforts  in  meeting 
system  requirements.  A  discussion  of  for¬ 
mal  reviews  specifically  detailed  in 
MIL-STD-499A  and  MIL-STD-1521  is  pro¬ 
vided. 


Key  elements,  insofar  as  the  acquisition 
engineer  is  concerned  are  as  follows: 

a.  Computer  program  functional  flow 
diagrams  should  be  periodically  reviewed 
and  actual  status  of  the  activities 
reflected  therein. 

b.  Requirements  allocation  sheets,  in¬ 
dicating  in  which  program  modules  vari¬ 
ous  ATE  and  TS  system  requirements  are 
allocated,  should  be  audited  periodi¬ 
cal  ly. 

c.  Program  milestones  for  such  activi¬ 
ties  as  preliminary  computer  program  re¬ 
quirements,  computer  program  design, 
flow  charting,  coding,  subsystem  check¬ 
out  and  software  integration  activities 
should  be  scheduled.  These  schedules 
should  be  kept  up  to  date  and  the  con¬ 
tractor's  actual  performance  against 
these  schedules  should  be  periodically 
reviewed  and  audited. 

d.  The  ATE  and  TS  acquisition  engi¬ 
neer  should  be  aware  of  trade  studies 
made  by  the  contractor  and  why  decisions 
based  on  these  trades  were  made  as  they 
were. 

e.  High  technical  risk  areas  should 
be  reviewed  frequently.  Particular  atten¬ 
tion  should  be  given  to  the  adequacy  of 
the  approaches  the  contractor  is  taking 
to  reduce  the  risk. 

f.  The  USAF  should  carefully  monitor 
software  language  and  computer  operating 
system  activities  of  the  contractor.  Com¬ 
pilers,  assemblers  and  operating  systems 
supplied  with  commercial  computers  of 
the  type  used  in  ATE  or  TS  are  normally 
maintained  by  the  computer  vendor.  It  is 
necessary  to  maintain  conf iguration  con¬ 
trol  of  these  software  elements  as  well 
as  the  application  software  being 
developed  by  the  contractor. 

g.  Detailed  review  and  audit  should 
be  conducted  by  the  USAF  of  such  things 
as  message  formats,  TRD  implementation 


by  ATLAS,  memory  utilization,  data  base 
design,  FORTRAN  use  or  machine  language 
coding  of  real  time  TS  algorithms,  etc. 
Through  detailed  knowledge  of  such 
things  and  knowing  where  the  contractor 
is  with  respect  to  his  plans,  the  acqui¬ 
sition  engineer  can  effectively  manage 
the  software  engineering  process. 

6.2  CONTRACT  CHANGES 

The  regulations  and  other  command  media 
previously  identified  are  equally  appli¬ 
cable  to  contract  changes.  Such  changes 
come  about  for  a  variety  of  reasons.  ATE 
and  TS  are,  in  many  cases,  the  "slave" 
of  the  weapon  system  of  which  the  ATE  or 
TS  is  a  part.  As  the  configuration  of  an 
airplane  evolves  or  is  altered,  so  must 
the  flight  simulator  be  altered.  As  an 
electronic  system  is  changed  the  UUTs  to 
be  tested  by  ATE  change  and  this  can 
have  significant  impact  on  the  ATE  being 
procured.  The  ATE  or  TS  acquisition  engi¬ 
neer  becomes  involved  in  this  change 
throughout  the  acquisition  process.  AFR 
800-12  identifies  as  high  change  sensit¬ 
ivity,  prototype  support  equipment  and 
that  which  is  obtained  as  deferred  devel¬ 
opment.  The  ATE  or  TS  acquisition  engi¬ 
neer  is  involved  in  the  change  process 
in  the  following  ways. 

6.2.1  Required  Changes 

The  acquisition  engineer  may  be  the 
first  to  be  aware  of  the  need  for  change 
and,  therefore,  the  first  to  communicate 
technical  change  requirements  to  the  con¬ 
tractor.  He  should  therefore  endeavor  to 
establish  and  maintain  effective  contrac¬ 
tor  communications  and  be  in  a  position 
to  discuss  software  technical  require¬ 
ments  in  detail  in  order  to  minimize  the 
cost  and  schedule  impact  of  changes. 

6.2.2  Required  Change  Impact 

The  acquisition  engineer  plays  a  heavy 
role  in  fact  finding  and  cost  negotia¬ 
tion  for  contract  changes.  Contractors 
are  not  normally  susceptible  to  the 
pressures  of  competition  for  contract 
changes.  Therefore  their  evaluation  of 


contract  change  Impact  will  often  be 
made  in  a  more  conservative  fashion.  It 
is  up  to  the  acquisition  engineer  to 
evaluate  the  reasonableness  of  contrac¬ 
tor  change  proposals  in  this  regard. 

6.3  CONTRACT  TERMINATION 

The  process  by  which  termination  of  a  TS 
or  ATE  contract  occurs  is  normally 
through  execution  by  the  PCO  of  a  form 
DD250.  However  the  acquisition  engi¬ 
neer’s  role  in  this  process  is  in  deter¬ 
mination  whether  the  ATE  or  TS  system 
meets  the  specifications  prepared  for 
it.  In  order  to  determine  that  the  con¬ 
tractor  has  complied  with  every  item  in 
the  software  specification,  the  acquisi¬ 
tion  engineer  will  determine  that  each 
specification  Section  4  software 
verification  requirement  has  been  met. 
Software  requirements  for  TS  and  ATE  are 
verified  by  test,  by  analysis  and/or  by 
inspection. 


a.  For  those  requirements  verified  by 
test,  the  acquisition  engineer  reviews 
test  directives  and  procedures,  test  re¬ 
sults  and  ensures  the  procedures  have 
been  satisfied. 

b.  For  those  verification  require¬ 
ments  satisfied  by  analysis,  the  contrac¬ 
tor  will  have  prepared  analysis  reports. 
The  acquisition  engineer  reviews  these 
reports  and  then  determines  technically 
whether  the  analysis  conducted  by  the 
contractor  verifies  the  particular  condi¬ 
tion. 

c.  Requirements  verified  by  inspec¬ 
tion  are  normally  associated  with  verifi¬ 
cation  that  certain  software  and 
documentation  has  been  supplied.  The  con¬ 
tractor  will  have  conducted  the  neces¬ 
sary  inspections  and  will  have  reported 
this  fact  through  contract  compliance 
channels.  The  acquisition  engineer  may 
be  called  upon  to  certify  adequacy  of 
these  inspections  and  to  determine 
through  his  own  inspection  whether  all 
software  items  required  by  the  contract 
have  been  delivered. 
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Section  8.0  MATRIX:  GUIDEBOOK  TOPICS  VS  GOVERNMENT  DOCUMENTS 

Figure  8.0-1  Is  a  cross  reference  matrix 
showing  the  guidebook  topics  and  govern¬ 
ment  documents  which  address  each  topic. 

Elements  In  the  matrix  Indicate  guide¬ 
book  sections  In  which  the  topic  Is 
discussed. 
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Figure  8.0 1.  Guidebook  Topics  Versus  Government  Documentation 


Section  9.0  GLOSSARY  OF  TERMS 


A1  gorithm  -  A  set  of  rules  or  processes 
tor  solving  a  problem  in  a  finite  number 
of  steps.  This  software  procedure  can  be 
presented  to  a  computer  precisely  and  in 
a  standard  form,  the  computer  then  takes 
the  algorithm's  course  of  action  to 
solve  the  problem. 

Armed  Forces  Procurement  Regulation  -  An 
interservice  publication  wnich  Ts  the 
basic  source-book  for  the  procurement 
process.  The  ASPR  is  the  "Bible"  for  all 
contracting  by  the  DOD  upon  which  Air 
Force  regulations  and  manuals  interpret 
and  implement  the  requirements  and 
policy  found  therein. 

Computer  Program  -  A  series  of  instruc¬ 
tions  or  statements  in  a  form  acceptable 
to  computer  equipment,  designed  to  cause 
the  execution  of  an  operation  or  series 
of  operations.  Computer  programs  include 
such  items  as  operating  systems,  utility 
programs,  and  maintenance/diagnostic  pro¬ 
grams.  They  also  include  applications 
programs  such  as  payroll,  inventory, 
control,  operational  flight,  strategic, 
tactical,  automatic  test,  crew  simulator 
and  engineering  analysis  programs.  Com¬ 
puter  programs  may  be  either  machine  de¬ 
pendent  or  machine  independent,  and  may 
be  general  purpose  in  nature  or  be  de¬ 
signed  to  satisfy  the  requirements  of  a 
specialized  process  of  a  particular 
user. 

Contract  -  A  legally  enforceable  agree- 
ment  between  two  parties  (AF /Contractor, 
Contractor/sub-contractor)  which  de¬ 
scribes  a  program  for  product  acquisi¬ 
tion.  The  contract  contains  the  System 
Specifications,  the  Statement  of  Work, 
the  Contract  Data  Requirements  List,  and 
the  Work  Breakdown  Structure. 

Control  Software  -  Software  used  during 
execution  of  a  test  program  which  con¬ 
trols  the  nontesting  operations  of  the 
ATE.  This  software  is  used  to  execute  a 
test  procedure  but  does  not  contain  any 
of  the  stimuli  or  measurement  parameters 
used  in  testing  a  unit  under  test.  Where 


test  software  and  control  software  are 
combined  in  one  inspeparable  program, 
that  program  will  be  treated  as  test 
software.  (AFLC  66-37) 

Data  Base  -  A  collection  of  program 
code,  tables,  constants,  interface  ele¬ 
ments  and  other  data  essential  to  the 
operation  of  a  computer  program  or 
software  subsystem. 

Life  Cycle  Analysis  -  An  analysis  of  a 
systems  total  cost  to  the  government 
over  its  full  life.  It  would  include  the 
cost  of  development,  production,  opera¬ 
tion,  support,  and  if  applicable, 
di sposal . 

Logic  Flow  -  A  diagrammatic  representa¬ 
tion  of  the  logic  sequence  for  a  com¬ 
puter  program.  Logic  flows  may  take  the 
form  of  the  traditional  flow  charts  or 
in  some  other  form  such  as  a  program 
design  language. 

Organic  -  A  term  used  to  designate  a 
task  performed  by  the  Air  Force  rather 
than  a  contractor. 

Software  -  A  combination  of  associated 
computer  programs  and  computer  data  re¬ 
quired  to  enable  the  computer  equipment 
to  perform  computational  or  control 
functions. 

Source  Selection  -  The  process  of  select¬ 
ing  which  among  competing  contractors 
shall  be  awarded  a  contract.  A  signifi¬ 
cant  portion  of  this  involves  evaluation 
of  proposals  to  determine  the  degree  to 
which  the  government's  requirements 
would  be  satisfied. 

Support  Software  -  Auxiliary  software 
used  to  aid  in  preparing,  analyzing  and 
maintaining  other  software.  Support  soft¬ 
ware  is  never  used  during  the  execution 
of  a  test  program  on  a  tester,  although 
it  may  be  resident  either  on-line  or  off¬ 
line.  Included  are  assemblies,  com¬ 
pilers,  translators,  loaders,  design 
aids,  test  aids,  etc.  (AFLC  66-37) 
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System  Engineering  -  The  application  of 
scientific  and  engineering  efforts  to 
transform  an  operational  need  or  state¬ 
ment  of  deficiency  into  a  description  of 
systems  requirements  and  a  preferred  sys¬ 
tem  configuration  that  has  been  opti¬ 
mized  from  a  life  cycle  viewpoint.  The 
process  has  three  principal  elements: 
functional  analysis,  synthesis,  and 
trade  studies  or  cost-effectiveness 
optimization. 

Test  Software  -  Programs  which  implement 
documented  test  requirements.  There  is  a 
separate  test  program  written  for  each 
distinct  configuration  of  unit  under 
test  (AFLC  66-37). 

Val idation  -  Computer  program  validation 
is  the  test  and  evaluation  of  the  com¬ 
plete  computer  program  aimed  at  ensuring 
compliance  with  the  performance  and 
design  criteria. 


Verification  -  Computer  program  verifica- 
tion  is  the  iterative  process  of  continu¬ 
ously  determining  whether  the  product  of 
each  step  of  the  computer  program  acqui¬ 
sition  process  fulfills  all  requirements 
levied  by  the  previous  step,  including 
those  set  for  quality. 

Work  Breakdown  Structure  -  A  standard 
method  ?or  structuring  a  program  i nto 
its  various  required  work  tasks.  A  Work 
Breakdown  Structure  is  implemented  per 
MIL- STD- 881 A  under  the  guidance  in  AFR 
800-17.  When  subdivided  as  necessary  by 
the  contractor  to  identify  tasks  asso¬ 
ciated  with  a  single  responsible  organi¬ 
zation,  it  provides  a  basis  for  contract 
planning,  status  determination,  and 
reporting. 
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Section  10.0  ABBREVIATIONS  AND  ACRONYMS 


AFLCP 

Air  Force  Logistics  Convnand 

DOD  I 

Department  of  Defense 

Pamphlet 

Instruction 

AFPRO 

Air  Force  Plant  Representative 

DSARC 

Defense  System  Acquisition 

Office 

Review  Coucil 

AFR 

Air  Force  Regulation 

ECP 

Engineering  Change  Proposal 

AFSCP 

Air  Force  Systems  Command 

ESD 

Electronic  Systems  Division 

Pamphlet 

HOL 

Higher  Order  Language 

AGE 

Aerospace  Ground  Equipment 

ICE 

Independent  Cost  Estimate 

ASPR 

Armed  Forces  Procurement 

Regulation 

O&M 

Operational  and  Maintenance 

ATE 

Automatic  Test  Equipment 

O&S 

Operational  and  Support 

ATLAS 

Abbreviated  Test  Language  for 

PCO 

Procurement  Contracting 

All  Systems 

Of f i cer 

CCP 

Contract  Change  Proposal 

PMD 

Program  Management  Directive 

CDR 

Critical  Design  Review 

RDT&E 

Research,  Development,  Test 

and  Evaluation 

CDRL 

Contract  Data  Requirements 

List 

RFP 

Request  for  Proposal 

CFSR 

Contract  Funds  Status  Reports 

ROC 

Required  Operational 

Capability 

CPAF 

Cost  Plus  Award  Fee 

SA 

Supplemental  Agreement 

CPFF 

Cost  Plus  Fixed  Fee 

SAE 

Software  Acquisition 

CPIF 

Cost  Plus  Incentive  Fee 

Engineering 

CPR 

Cost  Performance  Report 

SOW 

Statement  of  Work 

CPU 

Central  Processing  Unit 

SPO 

Systems  Program  Office 

CRT 

Cathode  Ray  Tube 

SSA 

Source  Selection  Authority 

DBMS 

Data  Base  Management  System 

SSAC 

Source  Selection  Advisory 

Council 

DCP 

Decision  Coordinating  Paper 

SSEB 

Source  Selection  Evaluation 

D&O 

Design  and  Development 

Board 

DID 

Data  Item  Description 

SSEC 

Source  Selection  Evaluation 

Committee 

DOD 

Department  of  Defense 

TRD 

Test  Requirement  Document 
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TS  Training  Simulator 

USC  United  States  Code 

UUT  Unit  Under  Test 

WBS  Work  Breakdown  Structure 
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3.2 

Contract  Definitization  and 

5. 2. 5. 3 

Price  Negotiations 

Proposal  Evaluation 

5.2.4 

Contract  Termination 

6.3 

Publicizing  Procurements 

5.1.2 

Cost  Plus  Award  Fee 

3. 1.1. 3 

Requests  for  Proposal 

4.0,  4.1 

Contracts 

Responsibilities  of  SSA, 

3. 2. 2. 2 

Cost  Plus  Fixed  Fee 

3.1. 1.3 

SSEC 

Contracts 

Selection  of  Contractors 

5.3 

Cost  Plus  Incentive  Fee 

3.1. 1.3 

Contracts 

Source  Selection  Advisory 

3.2.2. 1 

Council  (SSAC) 

Cost  Reimbursement  Contracts 

3. 1.1. 3 

Source  Selection  Authority 

3. 2. 2.1 

Data  Item  Descriptions 

4.2.3 

(SSA) 

Decision  Coordinating  Paper 

5.1.1 

Source  Selection  Evaluation 

3.2.2. 1 

Board 

DSARC 

5.1.1 

Source  Selection  Evaluation 

3.2.2 

Determination  of  Competitive 

5.2.5 

Committee 

Range 

Source  Selection 

3.2.2 

Fixes  Price  Incentive 

3.1.1. 3 

Organization 

Contracts 

Source  Selection  Plan 

3.2.3 

Formal  Source  Selection 

3.2.1,  5.1 

Process 

Source  Lists 

3.2.3, 

5.1.3, 

Four-Step  Procurement 

3. 1.1. 4 

5. 1.3. 2 

Statements  of  Work 

4.2.1 
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Special  Requirements  for  3.3 
Software  Acquisition 

Time  and  Materials  Contracts  3. 1.1. 3 

Two-Step  Formal  Advertising  3. 1.1. 2 
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